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Abstract 

In  today’s  reduction  of  America’s  national  defense,  campaign  level  models  are 
being  used  more  in  the  development  of  force  structure.  The  effects  of  drawdown  are  of 
significant  interest  to  those  at  the  highest  levels  of  authority.  Campaign  models  can  bring 
those  high  ranking  officials  the  answers  they  seek  with  high  confidence.  THUNDER  is  a 
campaign  model  used  frequently  by  the  United  States  Air  Force  and  many  of  its 
contractors.  The  effects  of  changing  the  force  structure  within  THUNDER  require 
modifying  variables  before  executing  a  new  experimental  run.  Changes  in  such  issues  as 
force  structure  cannot  be  immediately  addressed. 

Response  Surface  Methodology  (RSM)  can  be  used  to  provide  a  quick  answer  to 
effects  of  changing  force  structure  by  executing  several  experimental  runs  at  a  variety  of 
settings.  The  creation  of  a  “response  surface”  interlinks  each  scenario.  Factor  analysis  is 
a  multivariate  statistics  method  of  reducing  dimensionality  of  data  sets  and  determining 
relationships  between  measures  on  an  observation.  From  this,  relationships  can  be  found 
among  different  measures  of  effectiveness  to  create  new,  simpler  variables. 

The  methods  used  in  this  thesis  provide  a  means  for  creating  accurate,  “quick  turn” 
analysis  tools  which  a  decision  maker  can  use  to  make  timely  decisions. 
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THE  EFFECTS  OF  CHANGING 


FORCE  STRUCTURE  ON  THUNDER  OUTPUT 

I.  Introduction 

The  Department  of  Defense  (DoD)  compares  alternative  investment  and  policy 
choices  to  obtain  the  highest  level  of  effectiveness  possible  with  its  limited  resources. 
Recently,  modeling,  simulation,  and  analysis  have  become  an  integral  part  of  the  DoD 
decision  process.  Large  scale  computer  simulations,  ranging  from  high  resolution  one-on- 
one  scenarios  to  aggregate  models  of  large  scale  campaigns,  have  been  developed  to 
model  the  combat  environment.  These  simulations  provide  a  basis  for  comparing  the 
relative  effectiveness  of  alternative  weapons  systems,  force  structures,  operations 
concepts,  and  defense  policies.  Weapons  system  options,  for  example,  may  range  from  a 
small  modification  in  avionics  or  munitions  to  an  entirely  new  aircraft,  such  as  the 
Lockheed  F-22  Excalibur. 

The  number  and  type  of  aircraft  assigned  to  a  large  scale  campaign  can  be  a 
contentious  issue,  resulting  in  heated  debates  over  resources,  roles,  and  missions.  A 
typical  issue  concerns  the  relationship  between  the  air  forces  assigned  to  a  Joint  Force  Air 
Component  Commander  (JFACC)  and  the  effectiveness  of  the  resulting  theater-level  air 
campaign.  Air  Combat  Command  (ACC)  is  frequently  involved  in  these  type  of  issues. 
When  developing  its  positions,  the  Command  usually  employs  a  computer  simulation 
called  THUNDER.  Unfortunately,  THUNDER  generally  requires  hours  of  computer  time 
to  conduct  a  sufficient  number  of  simulation  runs  to  aid  in  the  staffing  and  planning 
processes. 
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Due  to  time  constraints  and  deadlines,  the  analysis  support  of  the  staffing  process 
rarely  has  enough  time  to  simulate  all  alternatives  under  consideration.  Command  analysts 
must  anticipate  a  variety  of  potentially  provocative  questions  and  develop  “quick  turn” 
tools  or  models  to  respond  to  them  in  a  timely,  accurate  manner.  This  thesis  explores  two 
possible  methods  of  this  “quick  turn”  analysis. 
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II.  Background 


Introduction 

This  chapter  provides  a  background  on  the  materials  and  methods  used  in  this 
thesis.  The  topics  covered  in  this  chapter  include  THUNDER,  aircraft  force  structure, 
response  surface  methodology  (RSM),  model  validation,  principal  components  analysis, 
and  factor  analysis.  The  material  presented  in  this  chapter  is  intended  to  familiarize  the 
reader  with  the  topics  covered  and  provide  references,  via  the  bibliography. 

THUNDER 

THUNDER  is  a  two-sided,  theater-level  combat  simulation  model  that  simulates 
air  and  ground  combat,  as  well  as  logistics.  An  aggregated,  deterministic  ground  war  is 
used  in  conjunction  with  a  detailed  stochastic  air  war  to  simulate  theater-level  conflict 
(TAC  THUNDER  Analyst’s  Manual,  1992).  While  ground  units  are  modeled  at  the 
regiment  and  division  level,  aircraft  sorties  are  modeled  individually. 

Simulating  the  same  theater-level  campaign  under  different  conditions,  such  as  the 
number  of  and  type  of  aircraft  available,  provides  the  basis  for  estimating  the  effect  of 
those  changes  on  measures  the  campaign  objectives  (Forsythe,  2-4). 

THUNDER  uses  over  sixty  files  to  define  the  simulated  campaign  scenario.  One 
data  file,  titled  squadron.dat,  provides  information  on  each  aircraft  used  in  the  simulation. 
Besides  the  number  and  type  of  aircraft  available,  squadron.dat  provides  the  mission  class 
(air  superiority,  ground  attack,  deep  strike,  multi-role,  jammer,  etc.),  sortie  rate,  days  in 
theater,  and  apportionment  of  aircraft  to  mission  classes.  THUNDER  takes  into  account 
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the  percentage  of  each  aircraft  allotted  to  each  mission  type  and  creates  an  appropriate  air 
plan  to  meet  defined  war  objectives  (TAC  THUNDER  Analyst’s  Manual,  1992). 

The  THUNDER  database  contains  the  information  for  type  of  scenario,  weapon 
systems,  terrain,  and  force  structure.  Analysts  using  this  model  have  the  option  of 
changing  the  database  in  order  to  study  the  effects  of  the  changes  in  plans,  tactics, 
weapons  systems,  and  force  structures.  This  thesis  examines  changes  only  in  the  force 
structure  portion  of  the  database.  The  unclassified  baseline  scenario  consists  of  a 
preemptive  Iraqi  attack.  The  scenario  posits  that  the  Iraqi  forces  penetrate  Saudi  Arabian 
territory,  forcing  the  allied  forces  mobilize  to  deter  the  attack.  Each  side  fights  to  meet 
predetermined  objectives,  measured  in  terms  of  attrition,  movement  of  the  forward  line  of 
troops  (FLOT),  and  the  time  needed  by  each  side  to  accomplish  objectives  (TAC 
THUNDER  Analyst’s  Manual,  1992). 

THUNDER  is  written  in  SIMSCRIPT®  II. 5.  It  requires  over  500  megabytes  of 
space  on  a  hard  disk  drive  and  can  be  run  on  any  machine  that  supports  SIMSCRIPT® 

II.  5.  A  SUN  or  DEC  workstation  is  required  to  support  the  terminal  graphics  used  in 
THUNDER’S  situation  map  and  grapher  (TAC  THUNDER  Analyst’s  Manual,  1992). 
Aircraft  Force  Structure 

THUNDER  models  aircraft  from  the  inventories  of  the  Air  Force,  Army,  Navy, 
and  Marine  Corps.  The  Air  Force  inventory  includes  the  Fairchild  A- 10  Thunderbolt  II, 
McDonnell-Douglas  F-15  Eagle,  Lockheed  F-16  Fighting  Falcon,  Lockheed  F-l  1 1 
Aardvark,  and  Lockheed  EF-1 1 1  Raven. 
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The  Fairchild  A- 10  Thunderbolt  II  is  a  ground  attack/close  air  support  aircraft. 
The  McDonnell-Douglas  F-15  Eagle  is  primarily  assigned  to  air-to-air  engagements; 
however,  the  F-15E  is  modified  for  ground  attack.  In  this  study,  the  F-15  was  strictly 
employed  as  an  air-to-air  aircraft.  The  Lockheed  F-16  Fighting  Falcon  is  a  “multi-role” 
aircraft  that  serves  as  both  an  air-to-air  and  ground  attack  aircraft.  The  Lockheed  F-l  1 1 
Aardvark  performs  deep  strike  missions  behind  enemy  lines.  The  sole  mission  class  of  the 
Lockheed  EF-1 1 1A  Raven  is  jamming,  and  the  aircraft  is  designated  to  perform  all 
jamming  mission  types.  The  Air  Force  aircraft  listed  were  the  only  aircraft  modified 
throughout  the  study.  The  study  scenario  included  other  Naval  and  enemy  aircraft,  but 
their  force  levels  were  not  modified  in  this  study. 

Response  Surface  Methodology  IRSM1 

RSM  consists  of  a  collection  of  statistical  techniques  for  empirical  model  building 
and  model  exploitation.  This  methodology  seeks  to  relate  a  response,  or  output  variable, 
to  the  levels  of  a  number  of  predictors,  or  input  variables,  that  affect  it  (Box  and  Draper, 
1).  The  set  of  outputs  or  responses  forms  a  response  surface.  Since  the  response  surface 
in  this  research  corresponds  to  the  output  of  simulation,  which  itself  is  a  model,  the 
response  surface  is  a  model  of  a  model,  or  a  metamodel  (Kleijnen,  1987:  147-148). 

In  estimating  a  response  surface,  the  true  response  function  is  unknown;  however, 
the  assumption  is  made  that  the  function  can  be  locally  approximated  by  a  polynomial  or 
some  other  type  of  function.  Designed  experiments  provide  the  data  needed  to  develop 
these  local  approximations. 
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Experimental  designs  induce  purposeful  changes  in  the  input  variables  in  order  to 
observe  changes  in  the  responses  (Box  and  Draper,  1987:17).  The  nature  of  the  design 
depends  on  several  factors: 

•  Which  input  variables  should  be  studied? 

•  Should  input  variables  be  transformed  and  then  examined? 

•  How  should  the  response  be  measured? 

•  At  which  levels  of  a  given  input  variable  should  experiments  be  run? 

•  How  complex  a  model  is  necessary? 

•  How  should  qualitative  variables  be  chosen? 

•  What  experimental  design  should  be  used?  (Box  and  Draper,  1987:  4-7) 

The  last  question  is  obviously  answered  by  the  first  six.  Available  resources  and  time 
available  should  also  be  considered  in  designing  an  experiment. 

The  information  needed  from  an  experiment  implies  the  minimum  acceptable 
resolution  for  the  design.  The  resolution  of  a  design  determines  the  degree  to  which  the 
estimates  of  factor  effects  will  be  aliased  or  confounded.  Two  or  more  factors  are 
confounding  if  their  effects  cannot  be  distinguished  from  one  another.  A  design  is  of 
resolution  k  if  all  n*  order  terms  are  not  aliased  with  any  other  terms  lower  than  order  k-n, 
where  n  <  k.  In  a  Resolution  III  design,  k=3  and  first  order  (n=l)  terms  are  aliased  with 
second  order  or  higher  terms  (k-n=2).  Resolution  III  and  Resolution  IV  designs  are 
considered  first  order  designs  since  no  first  order  terms  (or  main  effects)  are  aliased  with 
any  other  first  order  terms.  Although  first  order  terms  in  a  Resolution  IV  design  are  not 
aliased  with  any  terms  lower  than  third  order,  second  order  terms  can  be  aliased  with  other 
second  order  terms.  Designs  of  Resolution  III  and  IV  are  most  often  used  in  screening 
designs  to  determine  which  of  many  variables  under  consideration  are  important. 
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Resolution  V  designs  are  generally  used  for  establishing  how  the  input  variables 
affect  the  responses.  If  found  necessary  in  analysis,  this  design  resolution  is  ideal  for 
implementing  a  quadratic  model,  since  the  first  order  terms  are  not  aliased  with  any  terms 
lower  than  fourth  order,  and  the  second  order  terms  are  not  confounded  with  any  terms 
lower  than  third  order. 

The  designs  in  this  study  set  each  input  variable  at  either  its  high  or  its  low  level. 
These  levels  were  coded  as 
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(i) 


where  x,  is  the  coded  variable,  is  the  actual  variable  setting,  %t0  is  the  center  of  the 
region,  and  St  is  the  half-range,  or  half- width.  The  coded  value  for  the  high  level  is  +1, 
and  -1  for  the  low  level.  The  coded  design  points  and  responses  comprised  the  data  used 
to  generate  the  response  surface  metamodels. 

Multiple  linear  regression  techniques  were  used  to  create  a  response  surface  of  the 
simulation  output.  Several  statistical  tests  are  available  for  assessing  the  “goodness  of  fit” 
of  the  estimated  metamodel.  Most  of  these  measures  are  included  in  an  ANOVA  (analysis 
of  variance)  table,  or  a  regression  diagnostics  report.  The  coefficient  of  multiple 
determination,  or  R2,  expresses  the  percentage  of  variance  explained  by  the  model  (Neter, 
Wasserman,  and  Kutner,  241).  An  R2  value  of  one  means  the  model  fits  the  data 
perfectly,  or  100%  of  the  variance  is  explained.  Adding  terms  to  a  model  will  always 
improve  the  value  of  R2  (or  at  least  maintain  the  previous  value).  Since  R2  can  be  made 
large  by  the  simple  inclusion  of  additional  independent  variables,  a  modified  measure 
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adjusts  for  the  number  of  independent  variables  in  the  model.  The  adjusted  coefficient  of 
multiple  determination,  here  noted  as  “adj  R2”,  adjusts  R2  by  taking  into  account  the 
number  of  degrees  of  freedom,  or  observations,  in  the  model.  Taking  this  into  account 
provides  for  a  means  to  maintain  a  realistic  look  at  the  variance  explained  by  the  model. 

The  Mean  Square  Error  (MSE)  is  an  estimate  of  the  variance  of  the  model.  When 
the  model  is  properly  specified,  MSE  is  an  unbiased  estimator.  The  square  root  of  the 
MSE  is  the  standard  error,  and  by  definition  is  an  estimate  of  the  standard  deviation  of  the 
residual  error  of  a  model.  The  predicted  variance  of  a  model  is  larger  at  locations  furthest 
from  points  used  to  create  the  model.  The  maximum  predicted  variance  of  a  model  is 
found  with 

MPVar  =  (1  +  xT  ( XT  X)~ 1  x)  ■  MSE  (2) 

where  MPVar  is  the  maximum  predicted  variance,  X  is  the  coded  design  matrix,  and  x  is 
the  vector  of  maximum  coded  settings  (all  ones— these  settings  provide  the  greatest 
variability  of  the  model).  Here  again,  the  square  root  of  (2)  gives  an  indication  of  the 
maximum  predicted  standard  deviation.  This  measure  aids  in  judging  model  adequacy 
between  design  points. 

The  overall  F  statistic  is  a  measure  of  determining  if  the  output  is  a  function  of  any 
of  the  input  variables,  and  can  thus  be  considered  a  measure  of  adequacy  for  the  overall 
model.  Another  measure  of  statistical  significance  is  the  p-value,  or  attained  significance 
level.  This  value  is  the  smallest  level  of  significance  for  which  the  observed  data  indicates 
that  the  coefficient  in  question  is  not  actually  zero  (Mendenhall,  448).  In  other  words,  if 
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the  p-value  is  smaller  than  the  desired  level  of  significance  (noted  here  as  a),  then  the  term 
associated  with  that  p-value  is  significant  to  the  model. 

A  plot  of  the  residuals  versus  the  predicted  output  may  indicate  that  a  higher  order 
model  is  needed  for  the  data  used.  Finding  a  pattern  in  the  plot  may  indicate  a  better 
model  is  needed  to  describe  the  data.  A  plot  that  has  a  “shotgun  pattern”  does  not 
discredit  any  indications  the  model  is  adequate.  A  plot  having  a  diagonal  pattern  indicates 
a  poor  model,  and  another  attempt  at  modeling  should  be  made.  A  plot  that  forms  the 
pattern  of  an  arch  indicates  a  need  for  quadratic  or  cross-product  terms.  Finding  any  of 
these  patterns  indicates  a  violation  of  the  original  assumptions  (Neter,  Wasserman,  and 
Kutner,  1990:  116-121). 

The  use  of  these  statistical  measures  and  plots  adds  confidence  and  credibility  to 
model  adequacy.  If  a  linear  metamodel  is  not  adequate  for  the  data  used  in  the  model 
building,  a  quadratic  model  may  be  a  path  for  exploration  for  gaining  an  adequate  model. 
Augmenting  a  Resolution  V  design  with  additional  runs  that  incorporate  other  coded 
levels  proves  to  be  of  particular  value  in  determining  all  quadratic  and  other  second  order 
terms  in  the  model. 

A  central  composite  design  (CCD)  is  a  design  that  finds  quadratic  terms  by  using 
center  point  replications  and  “star,”  or  “axial,”  points.  Central  composite  designs  contain 
the  following: 

1 .  A  “cube,”  consisting  of  a  2k  factorial,  or  a  2k'p  fractional  factorial,  made  up  of 
points  of  the  type  (±1,  ±1,  ±1,  .  ..,  ±1),  of  resolution  R>  5  (Box  and  Hunter, 
1961a,  b)  replicated  rc  (  >  1)  times.  There  are  thus  nc  =  rc  2k'p  such  points 
(where  p  may  be  zero). 
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2.  A  “star,”  that  is,  2k  points  (±a,  0,  0,...,0),  (0,  ±a,  0,...,0),...,  (0,  0,  0,...,  ±a)  on 
the  predictor  variable  axes,  replicated  rs  times,  so  that  there  are  ns  =  2 krs  star 
points  in  all. 

3.  Center  points  (0,  0,  0,...,0),  n0  in  number,  of  which  nc0  are  in  cube  blocks  and 
ns0  in  star  blocks  (Box  and  Draper,  457). 

The  value  chosen  for  the  distance  from  the  center  of  the  design  to  a  star  point,  a, 

can  provide  for  the  model  aspect  of  rotatability.  The  value  of  a  is  usually  set  at  a  value 
which  is  greater  than  1,  thus  putting  the  star  points  outside  the  original  design  space.  The 
number  of  center  point  runs  determines  orthogonality.  With  some  designs,  having  these 
qualities  is  not  feasible.  These  qualities  are  also  not  always  essential.  While  desirable  to 
have  these  design  qualities,  options  such  as  the  “face  centered  central  composite  design” 
provide  for  a  means  to  determine  quadratic  terms  with  a  limited  operability  region.  This 
design  sets  the  value  of  a  to  be  set  at  the  limits  of  the  design  upper  and  lower  levels; 
namely  +1  and  -1.  Figure  2-1  shows  the  differences  in  design  for  a  central  composite 
design  and  a  face  centered  central  composite  design,  in  two  dimensions. 


Figure  2-1.  CCD  (left)  and  Face  Centered  CCD  (right) 
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When  the  value  of  a  for  rotatability  extends  beyond  the  design  region,  the  face-centered 
CCD  proves  to  be  a  acceptable  alternative,  provided  the  quality  of  rotatability  is  not 
important. 

Model  Validation 

Model  validation  serves  to  provide  confidence  in  the  final  model.  Three  basic 
methods  provide  a  means  for  validating  a  regression  model.  They  are: 

1 .  Collection  of  new  data  to  check  the  model  and  its  predictive  ability. 

2.  Comparison  of  results  with  theoretical  expectations,  earlier  empirical  results, 
and  simulation  results. 

3.  Use  of  a  hold-out  sample  to  check  the  model  and  its  predictive  ability.  (Neter, 
Wasserman,  andKutner,  1990:  465). 

The  best  means  for  model  validation  is  collecting  new  data.  This  new  data  could 
be  used  in  estimating  a  new  model  and  comparing  the  coefficients  of  the  validation  and 
original  model.  Credibility  is  added  to  the  model  if  the  coefficients  of  the  new  and 
validation  models  have  consistency. 

Metamodels  generated  in  this  thesis  are  validated  using  new  data  to  examine 
predictive  ability.  When  a  regression  model  is  developed  from  a  given  set  of  data,  the 
selected  model  is  inevitably  chosen  because  that  model  is  the  best  for  the  given  data.  A 
different  model  consisting  of  different  independent  variables,  interaction  terms,  and 
intercept  term  could  be  arrived  at  using  different  random  outcomes.  From  this  model 
development  process,  the  MSE  tends  to  understate  the  variance  of  the  predictive  ability  of 
the  selected  model  (Neter,  Wasserman,  and  Kutner,  1990:  465). 
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To  determine  the  actual  predictive  capability  of  the  selected  regression  model,  the 
model  is  used  to  predict  the  results  of  the  new  data  set.  Each  case  is  predicted  and  used  to 
estimate  mean  square  prediction  error. 

£  (T,  -  Y,y 

MSPR  =  -1=1 - - -  .  (3) 

n  * 

where  Y,  is  the  value  of  the  response  in  the  fth  validation  case,  is  the  predicted  value  for 
the  z'th  validation  case  based  on  the  model-building  data  set,  and  n*  is  the  number  of  cases 
in  the  validation  set. 

If  the  mean  squared  prediction  error  MSPR  has  a  value  fairly  close  to  the  model 
MSE,  then  the  regression  model  is  not  seriously  biased  and  gives  an  appropriate  indication 
of  the  predictive  ability  of  the  model.  If  the  MSPR  has  a  value  much  larger  than  the  MSE, 
the  mean  square  prediction  error  should  then  be  used  as  an  indicator  of  how  well  the 
selected  regression  model  will  predict  in  the  future  (Neter,  Wasserman,  and  Kutner,  1990: 
466). 

The  use  of  theory,  empirical  evidence,  or  simulation  results  also  serves  to 
determine  how  well  a  regression  model  will  predict.  If  a  data  set  is  large,  the  method  of 
“data  splitting”  can  be  used  to  create  a  “construct  data  set”  and  a  “validation”  or 
“prediction”  set.  This  procedure  is  also  called  “cross-validation”  (Neter,  Wasserman,  and 
Kutner,  1990:  466-467).  These  particular  validation  methods  are  not  used  in  this  research 
and  thus  are  not  discussed  in  detail  here. 
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Multivariate  Analysis 


Multivariate  Analysis  is  defined  as  the  application  of  methods  that  deal  with 
reasonably  large  numbers  of  measurements  (i.e.,  variables)  made  on  each  object  in  one  or 
more  samples  simultaneously.  Multivariate  analysis  deals  with  the  simultaneous 
relationships  among  variables.  While  univariate  and  bivariate  analysis  examines  the 
mean  and  variance  of  a  single  variable  or  a  pairwise  relationship  between  two  variables, 
multivariate  analysis  “examines  the  covariances  or  correlations  which  reflect  the  extent  of 
relationship  among  three  or  more  variables”  (Dillon  and  Goldstein,  1-2).  In  this  research 
effort,  the  multivariate  approaches  of  Principal  Components  Analysis  and  Factor  Analysis 
are  used. 

Principal  Components  Analysis  (PCA1 

Principal  Components  Analysis  “transforms  the  original  set  of  variables  into  a 
smaller  set  of  linear  combinations  that  account  for  most  of  the  variance  of  the  original  set” 
(Dillon  and  Goldstein,  24).  For  example,  several  measures  of  effectiveness  (MOEs)  in 
THUNDER  that  measure  Red  performance  may  be  combined  to  form  a  new  variable  that 
measures  how  well  the  Red  forces  performed  in  the  battle.  The  advantage  of  performing 
such  analysis  is  that  instead  of  tracking  four  or  five  of  the  original  MOEs,  one  variable  is 
observed  that  gives  an  overview  of  the  original  MOEs. 

One  of  the  first  considerations  in  PCA  is  to  decide  whether  a  covariance  matrix  or 
correlation  matrix  will  be  used  in  extracting  the  principal  components  from  the  data  set. 

If  the  variables  under  consideration  are  of  the  same  unit  of  measurement,  a  covariance 
matrix  is  acceptable.  If  the  variables  under  consideration  have  grossly  different  units,  the 
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composition  of  the  derived  components  can  be  influenced  by  scale  effects.  Hence,  the 
data  should  be  standardized  and  the  correlation  matrix  used  (Dillon  and  Goldstein:  1984, 
36). 

Once  the  data  have  been  standardized  and  a  correlation  matrix  calculated, 
eigenvalues  are  determined  from  the  correlation  matrix.  The  sum  of  the  eigenvalues.  A,, 
will  equal  the  number  of  variables,  p.  The  proportion  of  the  total  “variance”  explained  by 
each  component  is  Xjp.  (Dillon  and  Goldstein,  1984:  36). 

Interpreting  principal  components  is  clarified  with  the  use  of  component  loadings. 
The  first  column  of  the  loadings  matrix  describes  the  interrelationship  among  variables 
(MOEs)  for  the  first  principal  component.  The  jth  column  of  the  loadings  matrix  is 
associated  with  the  jth  largest  principal  component.  Fidell  and  Tabachnick  use,  as  a  rule 
of  thumb,  loadings  in  excess  of  0.3  as  eligible  for  interpretation.  It  is  further  suggested 
that  loadings  in  excess  of  0.71  are  excellent,  0.63  very  good,  0.55  good,  0.45  fair,  and 
0.32  poor  (41 1).  Higher  loadings  indicate  higher  correlations  with  other  loadings  with  a 
high  value  in  that  given  column.  The  actual  cutoff  for  interpretability  of  the  loadings  is  a 
matter  of  research  preference. 

Several  popular  techniques  exist  to  determine  how  many  principal  components  to 
retain  when  using  the  correlation  matrix.  Kaiser’s  criterion  suggests  retaining  those 
components  associated  with  eigenvalues  greater  than  one  (Fidell  and  Tabachnick,  1983: 
406).  Cattell’s  scree  test  uses  a  graphical  approach  where  each  factor/component  is 
plotted  (as  designated  (1,  2,  3,  etc.) )  against  the  value  of  its  respective  eigenvalue.  A 
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“scree  line”  is  then  applied  to  the  graph  to  separate  those  components  to  retain  (above 
line)  or  discard  (below  line)  (Dillon  and  Goldstein,  1984:  48-49). 

Once  the  decision  is  made  on  how  many  factors/components  to  retain,  component 
scores  may  be  generated  to  be  used  in  later  analyses.  These  scores  can  be  used  to  replace 
the  original  responses.  In  PCA,  these  scores  are  exact;  this  makes  this  approach  very 
appealing,  since  unique  scores  are  a  very  attractive  feature.  With  the  common  factor 
model,  having  unique  scores  is  generally  false  and  no  exact  solution  for  the  factors  is 
possible  (Dillon  and  Goldstein,  1984:  50).  Plotting  the  principal  components  scores 
against  one  another  can  provide  insight  to  trends  or  groupings  in  the  data  set. 

Factor  Analysis  (FA! 

Factor  Analysis  follows  the  same  procedures  as  PCA;  however,  fundamental 
differences  exist  between  the  two  approaches.  In  PCA,  unobservable  factors  are  functions 
of  its  indicators  (variables);  in  FA,  indicators  are  a  function  of  the  unobserved  factors. 

PCA  is  oriented  around  total  variation;  FA  is  oriented  around  common  variation.  Factor 
analysis  in  this  research  serves  to  clarify  the  results  of  principal  components  analysis. 
Factor  analysis  clarifies  PCA  results  by  rotating  the  axis  system  of  the  design  space.  The 
rotations  usually  performed  are  orthogonal.  With  an  orthogonal  rotation:  the  factors 
remain  uncorrelated,  the  variance  explained  by  a  specific  factor  changes,  and  the  total 
variance  explained  by  the  factors  remains  the  same,  but  each  factor’s  share  changes  (Fidell 
and  Tabachnick,  1983:  395-396).  The  most  popular  method  of  rotation  is  the  varimax 
rotation,  which  attempts  to  maximize  variation  of  squared  factor  loadings  within  a  factor 
(Dillon  and  Goldstein,  1984:  91).  Simply  put,  the  varimax  rotation  aims  to  make  any  large 
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correlations  larger,  and  any  small  correlations  smaller  (Fidell  and  Tabachnick,  1983:  387). 
Such  a  rotation  may  or  may  not  prove  useful  in  clarifying  the  principal  component 
loadings. 

Summary 

Applying  response  surface  methodology  to  investigate  the  effects  of  changing 
force  structure  in  THUNDER  leads  to  the  creation  of  metamodels  for  measures  of 
effectiveness.  Adequacy  of  these  metamodels  can  be  determined  with  test  statistics  and 
residual  plots.  A  metamodel  can  gain  purpose  and  credibility  through  validation  efforts, 
which  seek  to  determine  the  predictability  of  a  model  compared  to  new  data.  Data  with 
known  design  settings  can  be  applied  to  multivariate  analysis.  Using  multivariate  data 
analysis  may  serve  to  find  relationships  among  measures  of  effectiveness.  These 
relationships  can  then  be  reduced  to  more  general  terms,  which  provide  for  quicker, 
approximate  results. 
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III.  Methodology 


Introduction 

Chapter  III  covers  the  methodology  of  the  research  accomplished  in  this  thesis. 
First,  some  data  files  of  THUNDER  were  “competitively  enhanced”  so  as  to  make  the 
scenario  more  reasonable.  The  RSM  design  levels  (minimum  and  maximum  of  each 
aircraft  in  the  design  space)  were  decided  upon  with  input  from  the  customer,  HQ 
ACC/XP-SAS.  Response  surface  methodology  was  used  to  create  an  experimental  design 
and  create  metamodels  for  each  of  the  outputs.  Multivariate  analysis  was  then  performed 
using  the  methods  of  principal  components  analysis  and  factor  analysis,  discussed  in 
Chapter  II.  The  results  from  the  methodology  presented  here  are  discussed  in  Chapter  IV. 
Input  Variables 

The  input  variables  for  the  experimental  design  were  the  primary  tactical  aircraft  in 
the  United  States  Air  Force  inventory.  These  aircraft  included  the  A-10,  F-15,  F-16, 

F-l  1 1,  and  EF-1 1 1  A.  Only  the  number  of  each  aircraft  was  modified  in  the  research. 
These  modifications  were  made  in  the  squadrort.dat  file  of  THUNDER.  An  example  of  a 
squadron,  dat  file  is  found  in  Appendix  A. 

Output  Variables 

Five  Measures  of  Effectiveness  (MOE)  were  observed  as  part  of  this  thesis  effort. 
These  MOEs  included: 

•  The  number  of  days  needed  to  push  the  Iraqis  back  past  the  Forward  Line  of  Troops 
(FLOT)  set  at  day  one  (this  MOE  referred  to  as  “Days  needed  to  neutralize  FLOT”); 
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•  Days  needed  to  achieve  air  supremacy,  defined  as  the  time  when  the  Red  sortie  rate  is 
5%  or  less  than  the  day  one  sortie  rate; 

•  Number  of  air  kills  achieved  by  Blue  forces  of  Red  targets; 

•  Number  of  Blue  aircraft  lost,  and 

•  The  depth,  in  kilometers,  of  the  Red  forces’  advancement  into  Blue  territory. 

These  measures  were  collected  by  the  data  report  generated  by  THUNDER  at  the  end  of 
each  experimental  run. 

THUNDER  Modifications 

In  this  research,  an  unclassified  database  was  used  with  the  scenario  set  in  the 
Southwest  Asia  theater,  similar  to  Operation  Desert  Shield/Desert  Storm.  This  scenario, 
left  unchanged,  proved  to  be  too  unbalanced  to  show  any  importance  of  the  variables. 

The  war  objectives  were  met  in  a  matter  of  a  couple  of  days,  due  to  a  weak  Iraqi  response, 
and  thus  resulting  in  an  uninteresting  scenario.  Drastic,  unrealistic  changes  of  the  allied 
force  structure  would  have  been  needed  to  produce  noticeable  changes  in  output. 

Changes  were  made  to  both  sides  of  the  battlefield  to  make  the  wargame  more 
“competitive.” 

HQ  ACC/XP-SAS  furnished  a  scenario  where  the  Red  forces  “goals”  were  set 
further  into  Blue  territory  compared  to  the  baseline  scenario.  These  modifications 
influenced  the  movement  of  the  FLOT.  Several  of  the  data  files  were  modified  to  change 
the  war  objectives  of  the  Iraqis,  so  as  to  “force  the  action”  by  the  Iraqis  on  the  allied 
forces. 


3-2 


The  squadron.dat  file  was  further  modified  by  increasing  the  inventory  of  every 
Red  aircraft  squadron  by  50  percent.  The  European  Tornado  squadrons  were  deactivated 
to  reduce  the  size  of  the  allied  forces.  The  single  squadrons  of  the  F-l  17A  stealth  aircraft 
and  the  F-15E  ground  attack  aircraft  were  also  deactivated,  since  these  aircraft  have 
profiles  of  F-l  1 1  aircraft  in  the  unclassified  scenario.  Finally,  the  three  squadrons  of  the 
Navy  EA-6B  jammer  aircraft  were  deactivated  in  order  to  increase  any  importance  the 
EF-1 1 1 A  aircraft  had  to  the  simulation  outputs. 

All  of  these  adjustments  provided  the  stage  for  each  30  day  THUNDER  run.  Each 
run  was  performed  on  a  SUN  Sparc-2  workstation,  and  took  about  40  minutes  to  run  the 
program,  post-process  the  data,  and  generate  a  data  report. 

Linear  Design  with  Two-Factor  Interactions 

The  purpose  of  this  thesis  is  to  investigate  HOW  the  number  of  each  aircraft  affect 
the  five  selected  outputs.  The  initial  goal  was  to  create  a  model  that  would  contain  main 
effects  and  two-factor  interactions.  Given  these  criteria,  the  best  choice  for  the 
experimental  design  was  of  Resolution  V.  With  five  MOEs  under  study,  a  half-fraction, 

16  run  design  was  selected.  This  design  is  written  symbolically  as  2  v5'1  (Box  and  Draper, 
164).  Table  3. 1  is  the  uncoded  design  for  the  Resolution  V  design. 

The  lower  bound  for  each  aircraft  (except  for  the  EF-1 1 1  A)  represents  a  half 
squadron  of  aircraft,  while  the  upper  bound  represents  six  squadrons  of  aircraft  (Mehuron, 
1995:  48).  At  first,  the  aircraft  ranges  look  very  large;  however,  such  a  large  design  space 
will  generate  greater  ranges  in  the  measures  of  effectiveness  and  possibly  a  highly  irregular 
surface.  Highly  irregular  surfaces  are  not  modeled  well  by  two  level  designs.  Viewing  the 
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experimental  design  in  terms  of  squadrons,  the  range  is  from  0.5  to  6;  the  EF-1 1 1 A  would 
be  a  binary  variable.  The  Resolution  V  coded  design  used  in  experimentation  is  found  in 
Appendix  B. 


Table  3.1.  Uncoded  RSM  Design 


Run 

A-10 

F-15 

F-16 

F-lll 

EF-111A 

1 

12 

12 

12 

12 

24 

2 

144 

12 

12 

12 

0 

3 

12 

144 

12 

12 

o 

4 

144 

144 

12 

12 

24 

5 

12 

12 

162 

12 

24 

6 

144 

12 

162 

12 

0 

7 

12 

144 

162 

12 

0 

8 

144 

144 

162 

12 

24 

9 

12 

12 

12 

144 

0 

10 

144 

12 

12 

144 

24 

11 

12 

144 

12 

144 

24 

12 

144 

144 

12 

144 

0 

13 

12 

12 

162 

144 

24 

14 

144 

12 

162 

144 

0 

15 

12 

144 

162 

144 

0 

16 

144 

144 

162 

144 

24 

Center 

78 

78 

87 

78 

12 

After  performing  the  regression  analysis,  various  statistics  were  evaluated  to 
determine  the  adequacy  of  the  linear  and  two-way  interaction  model.  Any  metamodels  not 
having  a  strong  indication  as  adequate  only  as  a  linear  and  two-factor  metamodel  were 
considered  candidates  for  adding  quadratic  terms. 

Use  of  a  Face  Centered  Central  Composite  Design 

Chapter  II  discusses  the  measures  taken  to  adjust  the  experimental  design  for 
finding  any  significant  quadratic  terms.  The  experimental  design  for  finding  quadratic 
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metamodels  in  this  research  consisted  of  the  sixteen  original  design  runs,  three  center  point 
replications,  and  ten  runs  to  find  the  “star,”  or  “axial”  points. 

The  distance  from  the  center  of  the  design  to  the  axial  points  could  not  realistically 
be  set  so  as  to  promote  rotatability.  A  coded  value  of  ±2.38  would  have  been  necessary; 
however,  the  uncoded  low  value  corresponding  to  this  length  resulted  in  a  negative 
number  of  all  aircraft  in  the  experimental  design.  With  this  being  unrealistic,  ensuring  the 
metamodels  provided  a  uniform  distribution  of  information  was  not  guaranteed  (Box  and 
Draper,  1987:  488). 

Table  3.2  displays  the  settings  for  the  additional  runs  augmented  to  the  original 
sixteen  run  design.  Note  the  three  center  point  replications  correspond  to  runs  17-19, 
while  the  axial  points  correspond  to  runs  20-29. 

Table  3.2.  Additional  “Face  Centered  CCD”  Runs  to  Augment  Original  Design 


Run 

A-10 

F-15 

F-16 

F-lll 

EF-111A 

17 

78 

78 

87 

78 

12 

18 

78 

78 

87 

78 

12 

19 

78 

78 

87 

78 

12 

20 

144 

78 

87 

78 

12 

21 

12 

78 

87 

78 

12 

22 

78 

144 

87 

78 

12 

23 

78 

12 

87 

78 

12 

24 

78 

78 

162 

78 

12 

25 

78 

78 

12 

78 

12 

26 

78 

78 

87 

144 

12 

27 

78 

78 

87 

12 

12 

28 

78 

78 

87 

78 

24 

29 

78 

78 

87 

78 

0 
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With  this  design,  regression  analysis  generated  a  second-order  metamodel  only  if  two- 
factor  interactions  or  quadratic  terms  were  significant. 

Once  the  quadratic  models  were  calculated,  any  improvements  were  noted  with  the 
use  of  the  validation  data  set.  The  validation  results  were  calculated  with  both  the 
quadratic  metamodel  and  the  linear/two-factor  model.  Predictive  error  measurements 
were  then  compared. 

Multivariate  Analysis 

Multivariate  Analysis  was  performed  on  the  16  x  5  matrix  of  THUNDER  output. 
Two  similar  methods  of  analysis  were  used:  Principal  Components  Analysis  (PCA)  and 
Factor  Analysis  (FA). 

Principal  Components  Analysis 

The  PCA  approach  used  in  this  thesis  utilized  the  sample  correlation  matrix,  R,  to 
extract  principal  components,  since  the  population  covariance  matrix,  E,  was  not  known. 
The  matrix  R  was  also  selected  for  analysis  due  to  the  difference  in  units  between  each  of 
the  MOEs  (Fidell  and  Tabachnick,1983:  19).  Once  these  components  were  extracted, 
they  were  examined  to  determine  how  many  to  retain. 

Principal  Component  Scores  were  then  calculated  to  have  a  new  means  of 
representing  the  data.  The  scores  of  the  principal  components  were  then  plotted  one 
against  another  to  gain  insight  on  any  possible  trends  or  groupings. 

Factor  Analysis 

The  FA  method  used  in  this  thesis  was  accomplishing  an  orthogonal  rotation  of  the 
results  of  the  PCA.  The  varimax  rotation  was  used  to  change  the  axis  system  of  the 
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components  to  clarify  any  correlations  within  the  data.  The  number  of  factors  to  retain 
were  then  assessed,  and  from  this  correlations  between  the  MOEs  were  evaluated. 

Factor  Scores  were  then  calculated  to  see  if  any  of  the  insight  determined  in  the 
principal  components  analysis  could  be  clarified.  The  scores  of  the  retained  factors  were 
plotted  one  against  the  other.  Plotting  the  factor  scores  proved  to  be  helpful  in  explaining 
any  trends  or  groups. 

Summary 

Initial  investigation  with  THUNDER  revealed  modifications  to  the  simulation  were 
necessary  to  create  a  more  realistic  scenario.  Response  surface  methodology  provided  the 
method  of  collecting  data  points  and  the  framework  for  generating  metamodels.  Principal 
components  analysis  and  factor  analysis  presented  a  way  in  which  relationships  between 
inputs  and  outputs  were  seen  in  a  much  clearer  manner.  Creating  principal  components 
scores  and  factor  scores  provided  a  means  to  generate  plots  in  which  trends  and  groupings 
were  much  easier  to  extract.  Finding  these  relationships  between  the  inputs  and  outputs 
presented  results  that  could  be  used  to  gain  new  information  and  insight. 
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IV.  Results 


Output  Results 

Each  THUNDER  run  was  based  on  a  sequence  of  random  numbers.  THUNDER 
has  ten  different  random  number  “seeds”  which  generate  different  sequences.  Different 
seeds  were  used  for  each  run  of  the  campaign  model. 

The  sixteen  THUNDER  runs  of  the  initial  2V51  design  and  their  results  appear  in 
Table  4.1.  Initially,  the  number  of  blue  aircraft  lost  was  measured  by  percent  of  total  force 
for  an  experimental  run;  however,  using  this  measure  created  a  situation  where  the 
variance  of  the  response  was  a  function  of  the  value  of  the  response.  A  fundamental 
assumption  of  regression  is  that  the  variance  of  the  response  is  constant  throughout  the 
design  region.  Using  the  total  number  of  aircraft  lost  for  each  run  eliminated  this  problem 
(Forsythe,  3-6). 

Linear  and  Two-Factor  Interactions  Metamodels 

A  multiple  regression  analysis  was  performed  with  each  measure  of  effectiveness, 
using  the  statistics  software  Statistix  v4.1.  After  each  regression  was  performed,  the 
statistical  measures  discussed  in  Chapter  II  were  computed  to  evaluate  model  adequacy. 
The  most  significant  terms  were  then  chosen,  and  a  subsequent  regression  was  performed 
with  these  variables.  The  following  paragraphs  discuss  the  evaluation  of  each  response 
surface  generated. 

Appendix  C  contains  the  complete  regression  results  for  the  metamodels  of  the 
most  significant  terms.  The  results  of  these  regressions  also  appear  in  Table  4.2 
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Table  4.2  Response  Surface  Analysis:  Significant  Terms  Observed 
(Linear  and  Two-Factor  Interactions) 


Prediction 

Variable 

Days  to  Neutralize 
FLOT 

Days  to  Achieve 
Air  Supremacy 

#  of  Air 
Kills 

Number  of 
Blue  AC  Lost 

Depth  of  Enemy 
Advance 

Intercept 

13.5 

16.625 

9215.69 

115.438 

56.625 

A-10 

-0.5625 

1622.19 

9.6875 

-2 

F-15 

-0.125 

-1.25 

-4.6875 

F-16 

-0.5625 

-0.5 

2058.06 

8.8125 

4.125 

F-l  11 

-0.5 

256.063 

-1 

EF-1 1 1 A 

-0.125 

0.75 

147.937 

-3.6875 

A-10,  F-15 

0.1875 

3.0625 

A-10,  F-16 

A-10,  F-l  11 

-0.75 

4.0625 

A-10,  EF-111A 

0.3125 

-294.563 

5.3125 

0.625 

F-15,  F-16 

-176.188 

-4.8125 

F-15,  F-l  11 

-0.125 

-141.937 

6.4375 

F-15,  EF-1 11A 

-0.75 

F-16,  F-l  11 

F-16,  EF-1 11A 

0.3125 

0.875 

-183.688 

F-l  11,  EF-1 11A 

5.6875 

Adjusted  R-sq 

0.9306 

0.7627 

0.9931 

0.9446 

0.9489 

F  test  statistic 

29.71 

9.04 

270.86 

26.57 

47.47 

MSE 

0.0625 

1.16667 

52619.9 

21.5625 

1.33333 

Std  Error 

0.25 

1.08 

229.39 

4.644 

1.155 

Max  Predictive 

Std  Deviation 

0.348 

1.504 

319.4 

6.46 

1.61 

MOE  #1 :  Days  Needed  to  Neutralize  FLOT  (Neut  FLOT) 

The  A- 10,  F-15,  F-16,  and  EF-1 1 1 A  were  the  aircraft  found  at  a  90%  significance 
level  to  have  a  linear  effect  on  the  days  needed  to  move  the  Iraqis  back  into  their  territory 
completely.  The  primary  mission  of  the  A- 10,  and  one  of  many  missions  of  the  F-16,  was 
close  air  support  (CAS).  The  significance  of  the  F-15  variable  indicated  the  battle  for  air 
superiority  did  influence  the  ground  war.  The  F-l  1 1  contributed  through  an  interaction 
with  the  F-15.  The  EF-1 1 1A  presence  did  contribute  significantly  to  achieving  the 
objectives. 

Four  interaction  terms  appeared  in  the  final  metamodel.  Since  the  objective  of 


MOE  #1  is  to  minimize  the  number  of  days  needed,  negative  signs  on  these  terms  were 


expected.  However,  this  was  not  the  case,  and  a  better  understanding  of  why  some  of  the 
interactions  were  positive  was  needed. 

Analysis  of  interaction  terms  is  best  seen  visually  with  an  interaction  plot.  This 
type  of  plot  takes  into  consideration  the  possible  combinations  (high  and  low  levels)  and 
uses  the  average  output  from  each  combination  to  describe  graphically  the  interaction  over 
the  design  region.  One  of  the  interaction  terms  of  particular  interest  in  this  MOE  was  the 
term  involving  the  A- 10  and  F-15— a  combination  of  a  ground  attack  aircraft  with  an  air 
superiority  aircraft.  The  interaction  plot  of  this  term  appears  in  Figure  4.1. 


Interaction  Plot:  A-10  &  F-15 
Days  to  Neutralize  FLOT 


Figure  4.1  Interaction  Plot  of  A-10  and  F-15  for  MOE  #1 
(Optimum:  Minimize  Days) 

When  the  A-10  is  set  at  the  coded  high  level,  the  end  result  appeared  to  change  minimally 
in  changing  the  number  of  F-15  s.  This  change  seen  in  Figure  4. 1  is  within  the  error  of  the 
metamodel.  With  the  A-10  set  at  the  coded  low  level,  a  noticeable  difference  appeared 
between  having  many  or  few  F-15s.  This  difference  could  possibly  suggest  either  the 
importance  of  having  air  superiority  aircraft  flying  to  support  ground  operations,  or  the 
competition  for  resources.  In  a  battle  environment  where  aircraft  are  competing  for 
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resources,  an  optimum  balance  exists  for  aircraft  to  do  each  of  their  jobs  effectively, 
receive  appropriate  maintenance  and  support,  and  achieve  the  MOE  objectives  quickly  and 
decisively.  The  interaction  apparently  indicates  a  balance  of  the  aircraft  is  needed  to  do 
the  best  job  at  meeting  objectives.  The  interaction  term  acts  as  an  “adjustment  factor”  to 
the  linear  effects  for  these  shortcomings. 

The  F-15  and  F-l  1 1,  however,  had  similar  roles  in  that  their  mission  concerns  were 
not  on  the  ground  in  Blue  territory.  Figure  4.2  shows  the  interaction  plot  for  these  terms. 


Interaction  Plot:  F-15  4  F-111 
Days  to  Neutralize  FLOT 


Figure  4.2.  Interaction  Plot  of  F-15  and  F-111  for  MOE  #1 
(Optimum:  Minimize  Days) 


This  interaction  plot  does  show  the  importance  of  the  F-l  1 1,  despite  the  fact  it  was  not 
included  as  a  linear  term  in  the  final  metamodel.  With  the  F-111  set  at  the  low  coded 
level,  the  number  of  F-15s  present  appeared  to  not  make  a  significant  difference  in  the 
result.  When  the  F-l  1 1  was  coded  to  the  high  level,  a  significant  difference  is  seen 
between  having  few  or  many  F-l 5s.  With  the  F-15  coded  to  the  low  level,  a  much  better 
outcome  was  observed  compared  to  the  F-15  at  the  high  level.  This  difference  of  almost 
two  days  may  suggest  a  competition  for  resources,  or  the  importance  of  the  mission  of  * 
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deep  strike  interdiction.  The  F-15  (only  an  air  superiority  fighter)  may  contribute  little  to 
the  deep  strike  environment. 

MOE  #2:  Days  Needed  to  Achieve  Air  Supremacy 

The  F-15  was  the  primary  air  superiority  aircraft  among  the  five  aircraft  in  this 
study;  therefore,  this  aircraft  was  expected  to  have  the  most  influence  on  this  prediction 
function.  This  aircraft  did  have  the  most  importance.  Every  aircraft  did  contribute  in  the 
metamodel,  either  having  a  linear  effect  and/or  an  interaction  effect  with  another  aircraft. 

One  might  immediately  notice  that  the  EF-1 1 1 A  had  a  linear  effect  that  is 
detrimental  to  the  mission  of  achieving  air  supremacy.  One  possible  explanation  of  this  is 
found  in  the  mission  of  the  EF-1 1 1  A.  The  EF-1 1 1 A  was  “produced  for  missions  that 
include  barrier  standoff  jamming,  degradation  of  radars  during  CAS  operations,  and  close¬ 
injamming  and  direct  support  for  deep  strike  missions.”  During  Operation  Desert  Storm, 
EF-1 1 1A  area  jamming  was  crucial  to  maintaining  air  supremacy  (Mehuron,  140).  From 
its  list  of  missions,  the  EF-1 1 1 A  apparently  is  important  for  air  supremacy  maintenance, 
not  achievement.  Using  this  jammer  aircraft  to  support  achieving  air  supremacy  appeared 
to  lower  force  mix  effectiveness  in  accomplishing  air  supremacy  in  a  minimal  time. 

Consulting  a  THUNDER  expert  on  this  “quirk”  in  the  metamodel  provided  some 
useful  information.  Expert  advice  indicated  the  jammer  role  should  not  hinder  the 
accomplishment  of  air  supremacy.  Using  a  flight  of,  instead  of  zero,  aircraft  as  the  coded 
low  level  may  have  been  a  better  idea  for  the  experimental  design.  Recently,  THUNDER 
experts  indicated  having  trouble  modeling  measures  of  effectiveness  that  are  measured  in 
time  (Logan,  1996:  interview).  THUNDER  apparently  measures  time  in  an  abnormal 
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manner.  With  a  range  of  only  six  days  in  this  output,  this  unusual  time  measuring  may  be 
clouded  (Logan,  1996:  interview). 

The  EF-1 1 1 A  and  F-16  provided  an  interaction  effect  in  achieving  air  supremacy. 
Its  interaction  plot  is  found  in  Figure  4.3.  This  interaction  plot  demonstrates  part  of  the 
difficulty  in  explaining  the  presence  of  the  EF-1 1 1  A.  With  the  F-16  set  at  the  low  coded 
level,  the  presence  of  the  EF-1 1 1 A  did  appear  to  significantly  contribute  to  gaining  air 
supremacy  whether  actively  flying  in  the  scenario  or  not.  With  the  F-16  set  at  the  high 

Interaction  Plot:  F-16  &  EF-111A 
Days  to  Air  Supremacy 


Figure  4.3.  Interaction  Plot  of  F-16  and  EF-111A  for  MOE  #2 
(Optimum:  Minimize  Days) 


coded  level,  three  additional  days  are  needed  to  achieve  air  supremacy  objectives  with 
EF-1 1 1  As  flying.  Observing  sortie  rates  for  the  two  plot  points  under  the  F-16  coded  high 
level,  many  more  sorties  were  flown  for  CAS  with  the  EF-1 1 1  As  at  the  coded  low  level. 
This  suggests  the  importance  of  close  air  support,  as  well  as  a  competition  for  resources. 
Between  these  aircraft,  competition  for  resources  appears  to  be  a  significant  factor  to 
gaining  air  supremacy. 
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Another  interesting  interaction  was  that  of  the  A- 10  and  F-l  1 1.  The  A- 10  did  not 
have  a  linear  effect  on  achieving  air  supremacy,  as  this  aircraft  performed  CAS  missions. 
The  plot  of  this  interaction  is  found  in  Figure  4.4.  With  the  A- 10  at  the  coded  low  level, 
adding  F-l  1  Is  did  not  improve  the  days  needed  to  achieve  this  MOE;  in  fact,  adding 
F-l  1  Is  made  matters  worse.  With  the  F-l  1  Is  not  flying  to  perform  the  deep  strike 
mission,  no  aircraft  behind  enemy  lines  were  being  destroyed  on  the  ground.  These 
aircraft  became  eligible  to  participate  in  the  air  war,  and  thus  more  airborne  targets 


Interaction  Plot:  A-10  &  F-111 
Days  to  Air  Supremacy 


Figure  4.4.  Interaction  Plot  of  A-10  and  F-111  for  MOE  #2 
(Optimum:  Minimize  Days) 

for  the  F-l 5  and  F-l 6.  When  the  A-10  was  at  the  coded  high  level,  increasing  the  number 
of  F-l  1  Is  improved  mission  accomplishment.  A-lOs  in  mass  appeared  to  able  to  attack 
those  targets  aimed  at  Blue  aircraft— such  as  surface-to-air  missiles.  Those  aircraft 
performing  the  air  supremacy  mission  were  better  able  to  function  without  additional 
threats  from  the  ground.  Clearly,  these  two  aircraft  complemented  each  other  in  achieving 
air  supremacy. 
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MOE  #3 :  Number  of  Air  Kills 


Air  Kills  are  those  Red  targets  destroyed  by  Blue  aircraft.  The  majority  of  these 
targets  are  on  the  ground,  and  thus  ground  attack  aircraft  were  expected  to  be  the  major 
role  players.  The  A-10,  F-16,  and  F-l  1 1  performed  these  missions  and  did  make 
contributions  to  the  prediction  function.  The  EF-1 1 1  A’s  role  of  jamming  was  seen  as  a 
linear  term  and  an  interaction  with  the  F-16,  supporting  the  earlier  statements  describing 
the  EF-1 1 1  A’s  role  in  CAS  missions. 

Each  of  the  two-factor  interactions  in  this  metamodel  served  to  account  for  the 
finite  number  of  targets  and  multiple  aircraft  capable  of  destroying  those  targets.  These 
interactions  appeared  to  be  adjustment  factors  for  the  linear  effects.  The  EF-1 1 1 A  did  not 
destroy  targets,  per  se,  but  rather  provided  a  better  environment  for  target  destruction. 
The  interaction  plot  of  Figure  4.5  demonstrates  how  its  presence  contributes 


Interaction  Plot:  A-10  &  EF-111A 
Number  of  Air  Kills 


Figure  4.5.  Interaction  Plot  of  A-10  and  EF-111A  for  MOE  #3 
(Optimum:  Maximize  Kills) 


to  the  A-10  mission.  With  the  A-10  set  at  the  coded  low  level,  more  air  kills  are  seen  with 
the  EF-1 1 1 A  in  greater  number.  At  the  A-10  high  level,  the  principal  of  mass  becomes  ' 
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evident,  as  the  EF-1 1 1 A  presence  becomes  less  noticeable.  Some  targets  were  destroyed 
with  the  jammer  aircraft  present,  others  were  not. 

The  interaction  term  of  the  F-15  and  F-16  described  the  multi-role  aspect  of  the 
F-16,  as  less  F-15s  involved  in  the  war  meant  a  greater  demand  for  the  Fighting  Falcon  to 
fly  air-to-air  combat  missions.  The  interaction  term  of  the  F-15  and  F-l  1 1  indicated  that 
while  the  F-l  1 1  accomplished  its  deep  strike  missions,  the  presence  of  the  F-15  hindered 
the  ability  to  score  deep  strike  kills.  A  demand  for  resources  and  overlapping  missions  in 
the  same  airspace  were  possible  factors  that  lowered  the  overall  effectiveness  of  the  F-l  1 1 
to  score  deep  strike  kills.  The  comments  from  a  THUNDER  expert  examining  this 
interpretation  found  this  explanation  to  be  reasonable  (Logan,  1996:  interview). 

MOE  #4:  Number  of  Blue  Aircraft  Lost 

The  number  and  type  aircraft  lost  depends  on  the  lethality  of  the  mission  to  be 
performed,  as  well  as  how  well  equipped  the  aircraft  is  for  that  mission.  In  the  metamodel 
created  for  number  of  Blue  aircraft  lost,  those  aircraft  with  the  more  dangerous  missions 
played  a  major  role  in  calculating  this  measure  of  effectiveness. 

The  A- 10  and  F-16  with  their  CAS  missions  had  highly  significant  linear  effects. 
Having  more  of  these  aircraft  meant  more  possible  CAS  missions  to  be  flown,  and  thus 
more  opportunities  for  the  enemy  to  score  a  ground-to-air  kill.  The  EF-1 1  lA’s  jamming 
mission  can  be  accomplished  out  of  the  range  of  enemy  fire,  and  thus  its  linear  effect 
appeared  as  negative.  The  presence  of  the  F-15  in  the  metamodel  was  favorable  to  the 
Blue  forces,  as  the  F-15  most  likely  kept  the  skies  clear  of  enemy  aircraft,  as  well  as 
staying  clear  of  receiving  anti-aircraft  fire. 
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The  F-15  improved  survivability  of  other  aircraft;  an  example  of  this  is  seen  in  an 


interaction  plot  with  the  F-16  (Figure  4.6). 

Interaction  Plot:  F-15  &  F-16 
Number  of  Blue  Aircraft  Lost 


Figure  4.6.  Interaction  Plot  of  F-15  and  F-16  for  MOE  #4 
(Optimum:  Minimize  Aircraft  Lost) 

With  the  F-16  set  at  the  coded  low  level,  the  change  from  coded  low  to  high  F-15  is  not 
evident.  However,  at  the  F-16  coded  high  level,  the  F-15’s  presence  appears  to  preserve 
around  twenty  aircraft.  Flying  F-15s  in  higher  numbers  appears  to  make  the  battlefield 
environment  safer  for  all  other  aircraft.  The  THUNDER  expert  interviewed  agreed  with 
this  assessment  (Logan,  1996:  interview). 

The  jamming  mission  was  also  seen  as  important  through  an  interaction  plot  with 
the  F-l  1 1,  which  appears  in  Figure  4.7.  With  the  F-l  1 1  coded  to  the  low  level,  the 
presence  of  the  EF-1 1 1 A  appears  to  significantly  lower  Blue  aircraft  losses.  Setting  the 
F-l  1 1  at  the  coded  high  level  saw  more  aircraft  lost  and  a  less  significant  effect  of  the 
EF-1 1 1  A’s  presence.  The  role  of  the  EF-1 1 1 A  as  an  escort  jammer  displays  itself  well  in 
this  interaction;  escort  jamming  was  cited  as  important  by  the  THUNDER  expert 
consulted  (Logan,  1996:  interview). 
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interaction  Plot: 

F-111  &  EF-1 11A 
Number  of  Blue  Aircraft  Lost 


Figure  4.7.  Interaction  Plot  of  F-111  and  EF-111A  for  MOE  #4 
(Optimum:  Minimize  Aircraft  Lost) 

MOE  #5  ,  Depth  of  Enemy  Advance 

An  overview  of  the  entire  metamodel  for  determining  the  depth  of  the  Red  forces’ 
advance  into  Blue  territory  shows  all  aircraft  having  a  contributing  positive  effect  to  the 
Blue  forces.  The  F-16  had  the  most  significant  linear  influence,  most  likely  due  to  its 
multi-role  mission  status. 

Only  two  two-factor  interactions  were  present.  The  interaction  plot  of  the  A- 10 
and  EF-1 1 1 A  is  found  in  Figure  4.8.  With  this  plot,  it  is  clearly  seen  that  having  the 
maximum  number  of  EF-1 1 1  As  is  better  for  the  low  design  level  of  the  A-10,  and  less 
important  for  the  high  level.  The  plot  indicates  having  EF-1 1 1  As  present  enhances  the 
aircraft  force  mix.  Without  EF-1 1 1  As  flying,  Red  forces  appear  to  advance  further  on 
average.  Having  EF-1 1 1  As  flying  provides  a  favorable  effect  to  the  objective.  The 
EF-1 1 1 A  interacting  with  the  F-15  also  had  a  favorable  effect  to  the  objective. 
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Interaction  Plot:  A-10  &  EF-111A 
Depth  of  Enemy  Advance 


Figure  4.8.  Interaction  Plot  of  A-10  and  EF-111A  for  MOE  #5 
(Optimum:  Minimize  Kilometers) 


The  interaction  plot  for  The  F-15  and  EF-1 1 1 A  appears  in  Figure  4.6.  The 
interaction  plot  indicates  a  complementary  relationship  between  these  two  aircraft 


Interaction  Plot:  F-15  &  EF-111A 
Depth  of  Enemy  Advance 


Figure  4.9.  Interaction  Plot  of  F-15  and  EF-111A  for  MOE  #5 
(Optimum:  Minimize  Kilometers) 

in  the  mission  of  minimizing  the  depth  of  the  enemy  advance.  With  the  F-15  set  at  the 
coded  low  level,  having  EF-1 1 1  As  present  decreased  the  Red  advance  by  about  a  half 
kilometer.  Having  the  F-15  set  at  the  coded  high  level,  the  addition  of  the  EF-1 1 1 A 
squadron  aided  in  reducing  the  Red  advance  by  three  kilometers.  With  six  squadrons  of  ? 
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F-l  5s  in  the  force  mix,  adding  EF-1 1 1  As  did  little  to  improve  this  measure  of 
effectiveness.  The  F-l 5  appeared  to  need  support  from  the  EF-1 1 1 A  to  have  an  effect  on 
minimizing  the  depth  of  the  Red  forces’  advance;  or,  the  importance  of  the  EF-1 1 1 A  is 
seen  by  the  interaction  plot.  In  either  case,  the  THUNDER  expert  consulted  agreed  with 
the  importance  of  the  EF-1 1 1 A  in  this  objective  (Logan,  1996:  interview). 

Metamodels  with  Quadratic  Terms 

The  linear  and  two-factor  design  was  augmented  with  additional  runs  to  create  a 
design  for  finding  quadratic  terms  of  each  aircraft.  Each  MOE  was  tested  for  the  presence 
of  quadratic  terms;  only  two  MOEs  proved  to  have  quadratic  terms  of  a  90%  level  of 
significance.  Appendix  D  contains  the  complete  regression  output  results  from  Statistix, 
with  residual  plots  included.  The  coefficients  and  significant  statistics  are  presented  in 
Table  4.3. 

With  both  metamodels,  the  results  obtained  were  not  much  different  than  those  of 
the  linear  and  two-factor  models.  The  terms  most  important  for  number  of  air  kills 
remained  the  same,  with  the  addition  of  the  F-15  quadratic  term.  The  A-10/EF-1 1 1 A 
interaction  term  did  not  remain  a  part  of  the  depth  of  enemy  advance  metamodel  with  the 
addition  of  F-15  and  F-l  6  quadratic  terms  included.  Neither  metamodel  measured  up  to 
the  linear/two-factor  models  in  favorable  test  statistics;  however,  the  quadratic 
metamodels’  test  statistics  still  were  reasonable  for  an  adequate  metamodel.  Validation 
will  determine  which  is  the  better  predictor  of  the  pairs  of  metamodels.  Validation  also 
proves  useful  in  bringing  credibility  to  the  multivariate  statistics  results. 
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Table  4.3  Response  Surface  Analysis:  Significant  Terms  Observed 
(Quadratic,  Linear,  and  Two-Factor  Interactions) 


Predictor 

Number  of 

Depth  of 

Variable 

Air  Kills 

Red  Advance 

Intercept 

9015.45 

56.3905 

A-10 

1605.94 

-2.0000 

F-15 

-0.66667 

F-16 

2059.94 

-4.27778 

F-lll 

238.833 

-0.66667 

EF-111A 

107.167 

-0.94444 

A-10,  F-15 

A-10,  F-16 

A-10,  F-lll 

A-10,  EF-111A 

-294.563 

F-15,  F-16 

-176.188 

F-15,  F-lll 

-141.937 

F-15,  EF-111A 

-0.75 

F-16,  F-lll 

F-16,  EF-111A 

-183.688 

F-lll,  EF-111A 

A-10,  A-10 

F-15,  F-15 

229.157 

-2.14793 

F-16,  F-16 

2.35207 

F-lll,  F-lll 

EF-111A,  EF-111A 

Adjusted  R-sq. 

0.9866 

0.8677 

F  test  statistic 

230.10 

23.96 

MSE 

61395.3 

2.4152 

Standard  Error 

247.78 

1.554 

Max  Predictive 

1100 

2.178 

Standard  Deviation 

Principal  Components  Analysis 

The  results  from  this  analysis  consisted  of  determining  the  true  dimensionality  of 
the  data,  the  number  of  components  or  factors  to  retain,  and  an  analysis  of  those 
components  or  factors.  The  component  scores  and  factor  scores  were  examined  for  any 
possible  explanation  of  what  is  happening  with  interrelationships  among  the  variables; 
insight  was  gained  by  investigating  these  scores. 

To  select  the  correct  number  of  components  to  retain,  Kaiser’s  criterion  and 
Cattell’s  scree  test  was  used.  Kaiser’s  criterion  is  a  rule  of  thumb  in  which  any 
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components  or  factors  associated  with  an  eigenvalue  greater  than  one  should  be  retained. 
An  eigenvalue  less  than  one  would  explain  less  variance  than  one  of  the  original  variables. 
Cattell’s  scree  test  is  a  graphical  approach  compared  to  the  rubble  at  the  bottom  of  a  cliff. 
Any  components  or  factors  above  the  “rubble”  should  be  retained  (Dillon  and  Goldstein, 
1984:  48-50). 

The  data  matrix  used  for  the  principal  components  analysis  was  the  16x5  matrix 
of  the  THUNDER  output  results.  A  new  measure  of  effectiveness,  remaining  Red 
inventory,  was  used  instead  of  number  of  air  kills  in  the  multivariate  analysis  (reasons  for 
this  change  are  discussed  later).  The  calculations  for  the  principal  components  were  made 
using  Mathcad  PLUS  5.0  and  are  found  in  Appendix  E.  The  eigenvalues  that  determine 
the  amount  of  variance  explained  by  each  principal  component  were  extracted  from  the 
sample  correlation  matrix.  The  calculated  eigenvalues,  difference  between  each  and  the 
next  largest,  proportion  of  total  variance  explained,  and  cumulative  variance  explained  by 
the  principal  components  are  cataloged  in  Table  4.4. 


Table  4.4:  Eigenvalues  with  Relationships  Among  Them 


1 

2 

3 

4 

5 

3.3449 

0.8213 

0.6071 

0.1532 

0.0735 

Difference 

2.5236 

0.2142 

0.4538 

0.0797 

0.6690 

0.1643 

0.1214 

0.0306 

0.0147 

Cumulative 

0.6690 

0.8332 

0.9547 

0.9853 

1.0000 

Using  Kaiser’s  criterion,  only  the  first  principal  component  was  kept;  however,  the  second 
largest  eigenvalue  was  not  too  far  from  the  cutoff.  Figure  4. 10  is  the  scree  plot  of  the 
principal  components.  The  scree  plot  has  a  large  gap  between  the  first  and  second  , 
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Scree  Plot  of 
Principal  Components 
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Figure  4.10.  Scree  Plot  of  Principal  Components 
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principal  components,  thus  supporting  the  selection  of  only  one  principal  component.  The 
retained  principal  component  explained  66.9%  of  the  variance.  A  modified  loadings 
matrix  (loadings  modified  to  read  from  largest  to  smallest,  left  to  right)  from  Appendix  E 
is  shown  in  Table  4.5.  Note  that  the  top  row  of  the  matrix  contains  the  corresponding 
eigenvalue  to  that  principal  component.  Each  column  corresponds  to  a  principal 
component.  The  entries  in  the  first  numerical  column  suggest  some  very 


Table  4.5.  Loadings  Matrix,  L,  for  Principal  Components 


E-vals 

3.34490 

0.8213 

0.60708 

0.15323 

0.07351 

Nt  FLOT 

0.92191 

-0.17493 

0.27395 

0.20421 

-0.05222 

Air  Sup 

0.50518 

0.84396 

0.17965 

-0.01517 

-0.00468 

Red  EQ 

0.94020 

-0.16779 

-0.01875 

-0.13539 

-0.26304 

AC  lost 

-0.71817 

-0.10401 

0.68335 

-0.05492 

-0.05859 

Dpth  Ad 

0.91653 

-0.19860 

0.18011 

-0.10119 

0.27903 

PC  #1 

PC  #2 

PC  #3 

PC  #4 

PC  #5 

strong  correlations  between  the  MOEs  in  the  first  principal  component.  Days  needed  to 
neutralize  FLOT,  remaining  red  inventory,  and  depth  of  enemy  advance  are  very  highly 
correlated.  These  high  correlations  appear  to  indicate  better  equipped  Iraqis  penetrate 
further  into  Saudi  Arabia,  and  as  a  result,  more  time  is  needed  to  push  them  back  into  their 
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own  territory.  (The  initial  PCA,  using  number  of  air  kills  instead  of  remaining  red 
inventory,  produced  number  of  air  kills  very  oppositely  correlated  with  days  to  neutralize 
FLOT  and  depth  of  enemy  advance.  Since  the  correlations  are  at  almost  equal  levels,  a 
new  MOE  was  used  to  avoid  a  “canceling  out”  of  the  effects.)  Number  of  blue  aircraft 
lost  showed  itself  significant  in  the  first  principal  component,  having  an  opposite  response 
to  the  other  three  significant  correlations.  Blue  aircraft  lost  was  also  significant  in  the 
third  largest  principal  component,  with  an  opposite  sign.  The  correlation  in  the  largest 
principal  component  indicated  fewer  aircraft  were  lost  as  the  Iraqi  forces  were  more 
successful  in  achieving  the  objectives  of  the  other  three  MOEs.  The  further  the  Iraqis 
advanced,  the  fewer  the  number  of  Blue  aircraft  lost.  While  the  presence  of  this  high 
opposite  correlation  might  indicate  an  overrun  of  airfields,  consider  principal  components 
two  and  three.  These  components  fail  Kaiser’s  criterion  for  retention;  however,  having 
eigenvalues  close  to  one  suggests  a  rotation  of  the  axis  system  may  clarify  this.  Principal 
component  three  (labeled  “PC  #3”  below  the  appropriate  column),  indicates  an 
independent,  opposite  correlation  from  the  same  MOE  in  PC  #1.  This  apparent 
contradiction  was  clarified  with  a  rotation  of  the  axis  system,  which  is  discussed  later  in 
this  chapter. 

Days  to  air  supremacy  was  left  as  an  independent  indicator  in  PC  #2.  Column  two 
of  the  loadings  matrix  in  Table  4.6  corresponds  to  the  second  largest  principal  component, 
and  the  second  entry  in  the  column  locates  where  this  MOE  is  highly  correlated  to  nothing 
else  in  the  column.  Days  to  air  supremacy,  therefore,  continued  to  be  examined  as  an 
independent  measure. 
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Principal  component  1  can  be  used  to  create  a  linear  combination  of  all  the  outputs 
that  explains  66.9%  of  the  variance.  The  meaning  of  this  new  variable  was  examined  with 
principal  components  scores.  Principal  components  scores  were  generated  and  the  scores 
from  the  two  largest  principal  components  were  plotted  against  each  other.  These  scores 
were  found  from  the  calculations  in  Appendix  E.  The  plot  of  principal  components  scores 
appears  in  Figure  4.11.  The  three  center  point  replications  included  in  the  quadratic 
metamodel  experimental  design  were  “scored”  using  the  results  of  the  sixteen  run  study, 
and  are  included  in  the  principal  component  score  plots  (as  triangles)  to  see  how  these 
results  fared  against  the  others. 

The  plot  of  Figure  4.1 1  lists  aircraft  with  selected  points.  These  aircraft  were  set  at 
a  high  level  for  that  particular  experimental  run.  Notice  that  the  far  left  of  the  plot  consists 
of  points  with  only  one  aircraft  set  at  the  high  level.  At  the  far  right  is  the  point  with  all 
aircraft  set  at  the  high  level.  In  between  these  regions  are  the  various  combinations  of 
other  aircraft.  Not  observing  the  EF-1 1 1 A  high  level  (present  due  to  the  coding  scheme 
of  the  experimental  design),  the  runs  with  three  aircraft  high  are  more  to  the  right,  while 
the  runs  with  two  aircraft  high  are  to  the  left.  The  principal  component  scores  for  the  first 
principal  component  appear  to  indicate  the  best  force  mix  for  meeting  ground  war 
objectives.  A  lower  score  indicates  a  better  force  mix;  on  the  plot,  a  point  more  to  the 
right. 

If  any  evidence  is  to  not  support  this  conclusion,  the  A- 10  at  the  coded  high  level 
all  alone,  as  well  as  the  F-16  at  the  coded  high  level  alone  in  the  middle  of  the  plot  would 
suggest  these  aircraft  alone  are  better  than  some  combinations.  The  fact  that  the  stronger 
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Principal  Components  Scores: 

Principal  Component  1  vs.  Principal  Component  2 


Figure  4.11.  Principal  Components  Scores  with  Associated  High  Aircraft  Levels 

(Optimal:  to  the  Right) 


MOEs  within  the  component  are  more  oriented  to  the  ground ,  and  not  a  balance  of  air  and 
ground,  contribute  to  the  higher  scores  for  the  CAS  aircraft.  The  plot  overall  suggests  the 
A- 10  is  a  very  important  aircraft  to  the  force  mix.  The  F-16  is  also  important,  as  it  is  a 
multi-role  aircraft. 

Using  the  “spread”  of  the  center  points  in  the  plot  can  be  used  as  a  rough  rule  of 
thumb  to  distinguish  points  and  groups.  Any  points  separated  by  a  distance  greater  than 
the  largest  distance  between  center  points  may  have  distinguishing  characteristics.  Using 
this  rule,  the  far  left  of  the  plot  has  a  group  of  three  points,  while  the  far  right  has  a  group 
of  four  points.  The  rest  of  the  points  form  a  cluster  in  the  center  of  the  plot.  Each  of 
these  groups  could  be  seen  as,  from  left  to  right,  “below  average”  force  mixes,  “average” 
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force  mixes,  and  “optimal”  force  mixes.  Again,  the  force  mixes  to  the  right  appear  to  be 
best  suited  for  achieving  ground  war  objectives. 

Factor  Analysis 

A  varimax  rotation  of  the  axis  system  was  performed  in  an  effort  to  clarify  the 
loadings  in  Table  4.5.  Appendix  F  contains  the  SAS  output  (note  the  first  page  is  simply 
the  principal  components  results— the  factor  analysis  starting  point).  The  loadings  matrix 
resulting  from  rotation  appears  in  Table  4.6. 


Table  4.6.  Loadings  Matrix  after  Varimax  Rotation 


E-vals 

2.46566 

1.17091 

1.04928 

0.21594 

0.09821 

Nt  FLOT 

0.94378 

-0.17896 

0.18532 

0.07734 

-0.19218 

Air  Sup 

0.16624 

-0.13691 

0.97600 

0.03218 

0.00150 

Red  EQ 

0.78655 

-0.39841 

0.13677 

0.45125 

-0.01675 

AC  Lost 

-0.27956 

0.94554 

-0.15225 

-0.06750 

-0.00849 

Dpth  Adv 

0.92222 

-0.25955 

0.14308 

-0.02714 

0.24683 

FACTOR  1 

FACTOR  2 

FACTOR  3 

FACTOR  4 

FACTOR  5 

(Note  the  factors  now  appear  left  to  right  as  most  important  to  least  important.) 
The  loadings  matrix  now  shows  the  data  set  to  clearly  be  three  dimensional.  The  three 
highly  positively  correlated  MOEs  from  the  principal  components  analysis-days  needed  to 
neutralize  FLOT,  red  inventory  remaining,  and  depth  of  enemy  advance-were  again  highly 
correlated.  After  rotation,  the  consideration  of  Blue  aircraft  lost  in  the  first  principal 
component  was  removed.  This  first  factor  better  indicated  a  “ground  war  index”:  a 
successful  war  for  the  Blue  forces  constituted  a  low  index.  A  high  index  indicated  the  Red 
forces  were  more  successful  in  achieving  their  objectives.  Blue  aircraft  lost  and  days  to  air 
supremacy  remained  independent.  Using  Kaiser’s  criterion,  these  factors  are  retained. 
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Along  with  Kaiser’s  criterion,  a  scree  plot  of  the  factors  also  supported  keeping 
three  factors,  as  is  seen  in  Figure  4. 12.  With  three  factors  to  retain,  scores  of  these  factors 
proved  to  be  the  most  insightful. 

Scree  Plot  of  Factors 

♦  ♦ 

- 1 - 1 - * 

2  3  4  5 

Factor 

Figure  4.12.  Scree  Plot  of  Factors 

Factor  scores  were  computed  and  are  found  in  Appendix  E.  Three  plots  were 
created:  Factor  1  versus  Factor  2,  Factor  1  versus  Factor  3,  and  Factor  2  versus  Factor  3. 
Again,  these  plots  were  generated  to  gain  insight  as  to  the  meaning  of  the  Factor  Scores. 

The  plot  of  Factor  1  versus  Factor  2,  seen  in  Figure  4. 13,  shows  the  “ground 
objectives  index”  versus  the  “Blue  attrition  index.”  The  factor  scores  in  the  Ground 
Objectives  Index  had  a  similarity  to  the  principal  components  scores  in  that  they  appeared 
to  weigh  one  side  of  the  plot  with  individual  aircraft,  while  the  combinations  of  the  most 
aircraft  weighted  the  other  side.  To  better  understand  the  scores,  the  Ground  Objectives 
Index  was  exactly  that-concerned  with  those  war  objectives  dealing  with  the  ground.  The 
lower  factor  scores  (to  the  right  of  the  plot)  for  the  Ground  Objectives  Index  had  a  higher 
frequency  of  the  two  CAS  aircraft:  the  A- 10  and  F-16.  The  two  aircraft  not  involved  with 
CAS— the  F-15  and  F-l  1 1— were  found  as  part  of  higher  scores  more  often 
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Factor  Scores: 

Ground  Objectives  Index  vs.  Blue  Attrition  Index 


Figure  4.13.  Factor  Score  Plot  for  Factors  1  and  2,  with  High  Aircraft  Levels  Listed 

(Optimal:  Up,  Right) 


than  not  (left  of  the  plot).  The  center  point  scores  were  toward  the  center  of  the  plot,  and 
showed  a  wide  “spread”  in  the  ground  objectives  index.  Because  of  the  “spread”  in  center 
point  scores,  the  plot  indicates  no  distinct  groupings;  however,  the  index  scores  appeared 
to  indicate  how  good  a  force  mix  was  for  meeting  ground  war  objectives. 

Blue  aircraft  lost,  or  the  “Blue  Attrition  Index,”  was  an  independent  factor 
significant  after  rotation,  and  its  meaning  can  be  observed  in  both  plots  in  which  it  is 
included.  Moving  from  left  to  right  in  the  plot  of  Blue  Attrition  Index  versus  Air  Lethality 
Index,  found  in  Figure  4. 14,  the  CAS  aircraft  appeared  to  be  with  points  corresponding  to 
higher  scores,  while  the  air-to-air  and  deep  strike  aircraft  had  lower  scores.  From  this, 
higher  factor  scores  for  blue  aircraft  lost  suggested  the  primary  aircraft  used  were  under 
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Factor  Scores: 

Blue  Attrition  Index  vs.  Air  Lethality  Index 


Figure  4.14.  Factor  Score  Plot  for  Factors  2  and  3,  with  High  Aircraft  Levels  Listed 

(Optimal:  Up,  Right) 


the  CAS  mission;  lower  scores  involved  those  non-CAS  missions.  The  center  point  scores 
for  the  Blue  attrition  index  suggested  the  missions  accomplished  for  that  force  mix  were 
relatively  conservative;  CAS  missions  did  not  dominate  these  experimental  runs. 

A  lower  Air  Lethality  Index  meant  a  force  mix  which  would  produce  fewer  days  to 
achieve  air  supremacy.  Observing  Figure  4. 14,  the  upper  points  predominantly  consist  of 
the  F-15  and  F-16,  while  the  lower  points  consist  of  ground  attack  aircraft.  A  closer  look 
at  this  plot  suggests  two  groups,  divided  approximately  with  the  zero  of  the  Air  Lethality 
Index  axis.  The  upper  group  appeared  to  be  those  force  mixes  oriented  to  achieving  air 
supremacy.  The  lower  group  appeared  to  be  those  force  mix  combinations  not  suited  to 
achieving  air  supremacy  in  a  timely  manner.  Plotting  the  center  point  replications  indicates 
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the  points  in  the  extreme  upper  and  lower  regions  of  the  plot  are  those  force  mixes  with 
significant  differences  in  achieving  air  supremacy  quickly. 

An  expert  who  uses  THUNDER  daily  weighed  in  on  the  results  of  the  multivariate 
analysis,  and  found  them  very  favorable.  The  analysis  performed  here  would  prove  most 
interesting  with  the  classified  database  and  more  variables  to  correlate  (Logan,  1996: 
interview). 

Conclusions 

Using  principal  components  analysis  and  factor  analysis,  the  data  outputs  were  one 
dimensional  before  rotating  the  axis  system,  and  three  dimensional  afterwards.  The 
variable  generated  from  principal  components  analysis  demonstrated  a  measure  of  the 
principle  of  mass.  The  Ground  Objectives  Index  indicated  the  best  force  mix  consisted  of 
the  A-10,  F-15,  and  F-16  set  at  the  coded  high  levels.  The  Blue  Attrition  Index  prescribed 
a  force  mix  of  the  F-15,  F-16,  and  F-l  1 1  at  the  coded  high  levels  to  produce  the  fewest 
number  of  aircraft  lost.  The  Air  Lethality  Index  dictated  the  optimal  force  mix  of  the 
F-15,  F-16,  and  EF-111A. 

With  sixteen  runs  used  in  this  analysis,  a  validation  of  some  sort  would  provide 
confidence  and  credibility  in  these  results.  Data  points  collected  to  provide  validation  to 
the  RSM  results  of  the  research  were  also  used  to  provide  support  of  the  conclusions 
made  in  this  multivariate  analysis. 


4-25 


V.  Validation  of  Metamodels  and  Multivariate  Analysis 


Introduction 

In  Chapter  II,  some  methods  of  validating  a  statistical  model  are  explained. 
Validation  of  the  metamodels  generated  in  this  thesis  was  accomplished  with  the  collection 
of  new  data.  THUNDER  output  results  from  within  the  design  space  were  collected  and 
compared  to  the  outputs  of  the  metamodels  with  the  same  aircraft  settings.  These 
comparisons  were  then  used  to  measure  the  predictive  ability  of  the  metamodels.  The 
closeness  of  metamodel  prediction  to  actual  THUNDER  output  showed  how  successful 
and  useful  the  metamodels  could  be  to  a  potential  user. 

The  validation  experimental  runs  were  also  used  with  the  multivariate  analysis. 
Instead  of  beginning  a  new  study  with  a  data  set  of  sixteen  new  runs,  these  runs  were 
augmented  to  the  original  data  set,  and  an  analysis  of  32  runs  was  accomplished.  The 
multivariate  analysis  of  this  larger  data  set  hoped  to  clarify  and/or  solidify  any  deductions 
made  from  the  original  sixteen  runs. 

Experimental  Design  for  Collecting  Validation  Data 

The  strategy  for  collecting  validation  data  consisted  of  finding  observations  within 
the  original  design  region.  The  metamodels  generated  were  best  suited  for  the  design 
points  from  which  they  were  built.  Those  areas  between  the  design  points  are  where 
future  predictions  will  be  made;  rarely,  if  ever,  will  a  prediction  be  made  using  exact 
aircraft  levels  from  one  of  the  experimental  design  runs. 

A  half  fraction  factorial  design  was  used  to  systematically  collect  observations 
inside  the  design  region.  The  validation  design  coding  scheme,  simply  put,  was  the 
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original  design,  which  used  -1  and  +1  levels,  modified  such  that  the  levels  are  -0.5  and 
+0.5,  respectively.  The  uncoded  design  and  the  THUNDER  output  results  are  found  in 
Table  5.1. 

Results 


The  ability  of  the  metamodels  to  predict  outcomes  with  different  aircraft  force 
mixes  was  measured  with  the  Mean  Square  Prediction  Error  (MSPR),  discussed  in 
Chapter  II,  and  the  Mean  Absolute  Percentage  Error,  MAPE.  This  predictive  measure  is 
found  using 


MAPE  = 


(4) 


where  Y,  is  the  actual  observation  of  the  zth  validation  run,  Yt  is  the  predicted  value  by  the 
metamodel  for  the  zth  validation  run,  and  n  is  the  number  of  runs. 

Appendix  G  presents  the  validation  results  for  finding  MSPR  and  MAPE  using 
Microsoft  Excel  5.0.  Table  5.2  catalogs  the  results  of  the  MSPR  for  each  MOE  and 
compares  it  with  the  MSE  of  each  linear/two-factor  metamodel.  Recall  that  if  the  MSPR 
and  MSE  are  fairly  close  to  one  another,  the  metamodel  is  not  seriously  biased  and  gives 
an  appropriate  indication  of  the  predictive  ability  of  the  model.  If  the  MSPR  is  much 
larger  than  the  MSE,  MSPR  should  be  used  to  gage  how  well  the  metamodel  will  predict 
in  the  future  (Neter,  Wasserman,  and  Kutner,  1990:  466).  Table  5.2  also  shows  the  values 
for  the  MAPE  of  each  linear/two-factor  metamodel. 
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Table  5.2.  Comparison  of  MSE  and  MSPR,  MAPE  Results  for  Validation  Data 

(Linear/Two-Factor  Metamodel) 


MOE 

MSE 

MSPR 

MAPE 

Days  to  Neutralize  FLOT 

0.0625 

0.1232 

1.78% 

Days  to  Air  Supremacy 

1.1667 

8.68 

15.75  % 

Number  of  Air  Kills 

52619.9 

297944.6 

4.39  % 

Blue  Aircraft  Lost 

21.563 

99.75 

7.20  % 

Depth  of  Enemy  Advance 

1.3333 

3.6763 

2.73  % 

The  MSPR  of  each  measure  of  effectiveness  was  at  least  two  times  larger  than  the 
respective  MSE.  The  MSPR  appeared  to  be  a  measure  how  well  the  metamodel  will 
predict  in  the  future;  however,  for  the  first  and  last  MOEs,  the  difference  may  be  small 
enough  to  consider  the  predictive  ability  of  the  metamodels  to  be  unbiased.  The  MAPE 
indicated  these  metamodels  may  not  be  as  bad  as  the  MSPR  portrayed.  Only  the  number 
of  days  to  air  supremacy  had  a  MAPE  above  10  percent.  Such  a  significantly  higher  value 
for  this  measure  compared  to  the  other  four  may  be  due  to  the  fact  that  out  of  the  five 
MOEs,  this  one  had  the  most  subjectivity  associated  with  it.  Air  Supremacy  was  roughly 
defined  in  this  research  as  the  point  in  time  when  the  Red  forces’  aircraft  sortie  rate 
dropped  to  five  percent  of  the  day  one  sortie  rate.  The  values  recorded  were  the  result  of 
examining  data  reports  and  finding  the  first  day  of  many  when  the  sortie  rate  was  at  five 
percent  or  below  of  the  original;  or,  if  this  did  not  happen,  where  Red  sortie  rate  appeared 
to  stabilize  at  a  constant  rate.  The  MAPE  for  days  to  air  supremacy  would  possibly 
reduce  with  more  distinct,  defined  criteria  for  this  MOE. 

The  MSPR  and  MAPE  were  also  found  for  the  design  points—points  from  which 
the  metamodel  was  built.  These  results  appear  in  Table  5.3.  (Calculations  for  MSPR  and 
MAPE  for  both  validation  data  set  and  design  data  set  are  found  in  Appendix  G). 


5-4 


Table  5.3.  Comparison  of  MSE  and  MSPR,  MAPE  Results  for  Design  Data 

(Linear/Two-Factor  Metamodel) 


MOE 

MSE 

MSPR 

MAPE 

Days  to  Neutralize  FLOT 

0.0625 

0.15985 

2.396  % 

Days  to  Air  Supremacy 

1.16667 

1.519531 

5.893  % 

Number  of  Air  Kills 

52619.9 

19935.5 

1.301  % 

Blue  Aircraft  Lost 

21.5625 

7.8125 

1.998% 

Depth  of  Enemy  Advance 

1.33333 

2.011719 

2.128% 

The  MSPR  results  showed  the  metamodel  was  a  better  predictor  for  the  design  points  than 
for  the  validation  points.  Number  of  Air  Kills  and  Blue  aircraft  lost  have  a  smaller  MSPR 
than  MSE.  This  was  due  to  the  small  differences  in  prediction  and  actual  values.  A  trend 
was  seen  in  the  MAPEs,  in  comparison  to  the  validation  MAPEs.  Days  to  Air  Supremacy 
was  again  seen  having  the  highest  MAPE,  which  was  due  to  the  subjectivity  of  the 
measure  as  discussed  earlier.  The  rest  of  the  MAPEs  were  well  under  10  percent.  All  of 
the  MSPRs  for  the  design  data  were  not  significantly  larger  than  the  MSE  and  were  thus 
“not  seriously  biased”  in  giving  an  indication  of  the  predictive  ability  of  the  metamodels. 
The  amount  larger  than  the  MSE  was  relative  to  the  size  of  the  MSE;  while  “Number  of 
Air  Kills”  had  a  high  MSE,  its  MSPR  was  5.7  times  larger. 

Comparison  of  Linear/Two-Factor  and  Quadratic  Metamodels 

Table  5.4  compares  the  MSPR  of  the  Linear/Two-Factor  metamodels  and  the 
quadratic  metamodels.  Regarding  MOE  #3,  Number  of  Air  Kills,  the  MSPR  for  both  the 
validation  and  design  points  was  larger  than  for  the  linear/two-factor  metamodel.  This 
difference  indicates  the  linear/two-factor  metamodel  was  less  biased  than  the  quadratic 
metamodel,  and  thus  a  better  predictor  with  respect  to  this  predictive  measure.  The  same 
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can  be  said  for  the  validation  design  points  of  MOE  #5;  however  the  difference  is  less  than 
0.5. 


Table  5.4.  Comparison  of  MSPR  for  Linear/Two-Factor 
and  Quadratic  Metamodels 


MOE 

MSE 

Linear ZT\ wo-  Way  MSPR 

Polynomial  MSPR 

MOE  #3:  Val  Pts 

52619.9 

297944.6 

342473.9 

MOE  #3  :  Des  Pts 

52619.9 

19935.5 

26052 

MOE  #5:  Val  Pts 

1.3333 

3.6763 

4.1045 

MOE  #5:  Des  Pts 

1.3333 

2.0117 

1.1529 

The  design  points  revealed  an  MSPR  that  was  less  than  for  the  design  points.  The  two 
metamodels  appeared  to  be  almost  inseparable  in  predictive  ability  with  respect  to  MSPR. 
Table  5.5  compares  the  MAPEs  of  both  metamodels: 


Table  5.5.  Comparison  of  MAPE  for  Linear/Two-Factor 
and  Quadratic  Metamodels 


MOE 

Linear /P wo-  Way  MAPE 

MOE  #3:  Val  Pts 

4.385  % 

4.8595  %  j 

MOE  #3  :  Des  Pts 

1.301  % 

1.3175% 

MOE  #5:  Val  Pts 

2.727  % 

2.9030  % 

MOE  #5:  Des  Pts 

2.128% 

1.4343  % 

The  differences  in  MAPEs  for  the  two  metamodels  was  less  than  0.5  percent.  Even  for 
the  MAPEs  of  MOE  #5,  the  differences  were  less  than  one  percent.  In  general,  the 
quadratic  metamodels  did  not  do  any  better  at  predicting  than  the  linear/two-factor 
metamodels.  Even  if  the  quadratic  metamodels  had  been  better,  a  significantly  better 
predictive  ability  would  have  needed  to  be  seen  to  justify  the  additional  thirteen  runs. 
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How  much  confidence  put  into  the  metamodels  generated  depends  on  the  one  who 
uses  it  extensively  in  any  decision  making  process.  The  comparison  of  prediction 
measurements  over  the  design  space  and  validation  space  indicate  a  difference  of  less  than 
ten  percent  for  each  respective  MAPE.  The  decision  maker  must  decide  whether  or  not 
he  or  she  is  willing  to  have  up  to  a  ten  percent  error  in  the  approximations. 

Multivariate  Analysis  with  Both  Data  Sets 

Both  the  original  data  set  of  outputs  and  validation  set  of  outputs  were  combined 
and  a  principal  components  analysis  performed.  Appendix  H  contains  the  complete 
calculations  of  this  analysis. 

The  underlying  dimensionality  of  the  combined  output  set  showed  itself  to  have  the 
same  dimensionality  as  the  design  output  data  set.  The  correlations  within  each  principal 
component  were  very  similar  to  what  was  calculated  with  the  original  sixteen  runs.  The 
same  correlations  were  seen,  and  at  about  the  same  degree  of  correlation. 

The  initial  hypothesis  that  the  first  principal  component  reflected  the  ability  of 
achieving  ground  objectives  was  seen  in  much  clearer  in  a  plot  of  the  principal  components 
scores,  seen  in  Figure  5.1.  (Those  labels  with  a  “(V)”  indicate  the  run  was  a  validation 
run.  Those  points  which  are  triangles  are  center  point  replications.)  The  smallest  values 
for  the  first  principal  component  indicated  better  force  mixes  for  achieving  ground 
objectives.  The  absolute  lowest  first  principal  component  score  corresponded  to  all 
aircraft  at  the  coded  high  level.  The  highest  score  corresponded  to  the  experimental  run 
where  the  F-l  1 1  deep  strike  aircraft  was  the  only  aircraft  flying  at  the  coded  high  level. 

This  aircraft  does  not  actively  participate  in  ground  war  efforts  on  the  allied  side  of  the 
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FLOT.  The  next  highest  score  corresponded  to  only  the  F-15  air  superiority  fighter  flying 
at  the  coded  high  level.  After  the  points  referring  to  the  original  design  runs,  plotted 
scores  corresponding  to  the  validation  runs  with  similar  aircraft  levels  (for  example,  the 


Validation  Principal  Components  Scores: 
Principal  Component  1  vs.  Principal  Component  2 


5432  10-1-2-3-4 


Principal  Component  1 

Figure  5.1.  Validation  Principal  Components  Scores  with 
Associated  High  Aircraft  Levels 
(Optimal:  to  the  Right) 

validation  run  with  the  F-l  1 1  as  the  only  aircraft  at  the  coded  high  level)  appeared  next. 

Three  center  point  replications  were  scored  and  plotted  as  with  the  original  data 
set,  and  are  indicated  as  triangles  on  the  plot.  Their  relatively  close  scores  indicated  a 
small  variability  in  the  scores  of  principal  component  one.  This  close  variability  led  to 
three  distinct  regions  on  the  components  plot.  The  center  region  of  many  points  indicated 
force  mixes  which  were  very  similar  in  ability  to  address  ground  war  objectives.  To  the 
right  of  this  region,  these  aircraft  force  mixes  appear  to  be  above  average  to  optimum.  * 
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(Within  this  region,  the  validation  scores  were  higher  than  the  original  analysis  scores.) 

To  the  left,  aircraft  combinations  appeared  to  be  less  effective  in  the  ground  war.  (Within 
this  region,  the  validation  scores  were  lower  than  the  original  design  scores.)  The  “ground 
objectives  index”  appeared  to  indicate  a  smaller  score  means  a  better  aircraft  force  mix  for 
addressing  ground  war  objectives. 

After  rotating  the  loadings  matrix  (SAS  output  of  this  found  in  Appendix  I),  the 
Ground  Objectives  Index  appeared  to  have  this  quality  of  grouping  aircraft  mixes  in  a 
more  distinct  manner.  All  aircraft  force  mixes,  except  for  two,  fell  into  one  group  in  the 
center  of  the  plot  of  Ground  Objectives  Index  versus  Blue  Attrition  Index,  found  in  Figure 
5.2.  The  low  score  outlier  consisted  of  the  force  mix  of  the  A-10,  F-15,  and  F-16.  This 
appeared  to  indicate  that  having  all  of  these  aircraft  alone  set  at  the  coded  high  level, 
dedicating  all  resources  to  them,  will  result  in  the  most  decisive  achievement  of  MOEs 
pertaining  to  the  ground  war.  The  other  extreme  point  was  the  run  corresponding  to  the 
F-l  1 1  coded  at  the  high  level,  suggesting  this  force  mix  was  the  least  effective  in  achieving 
ground  war  objectives. 

The  variability  in  the  center  point  replications  indicated  these  outlier  end  points 
were  not  extremely  different  from  the  respective  edges  of  the  central  cluster  of  plotted 
points.  Because  of  this  variability,  the  Ground  Objectives  Index  appeared  to  rank  order 
the  various  force  mixes,  from  most  to  least  effective.  This  clarified  the  initial  results  with 
the  principal  components  scores.  The  plot  of  Figure  5.2  also  clarified  the  meaning  of  the 
Blue  Attrition  Index. 
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Factor  scores  for  the  Blue  Attrition  Index  appeared  to  indicate  a  high  score  meant 
more  aircraft  lost  in  combat.  The  highest  factor  score  consisted  of  the  experimental  run 
where  only  the  F-16  flew  in  high  numbers;  the  lowest  factor  score  consisted  of  the 
experimental  run  where  the  F-15,  F-16  and  EF-1 1 1 A  flew  in  high  numbers.  Referencing 
Table  4.2,  the  run  with  only  the  F-16  flying  in  high  numbers  lost  155  aircraft— the  most  of 


Validation  Factor  Scores: 

Ground  Objectives  Index  vs.  Blue  Attrition  Index 


Figure  5.2.  Validation  Factor  Scores: 

Ground  Objectives  Index  versus  Blue  Attrition  Index 
(Optimal:  Up,  Right) 


any  run.  Similarly,  the  run  with  only  the  F-15,  F-16,  and  EF-1 1 1 A  flying  in  high  numbers 
lost  87  aircraft-the  least  of  any  run.  These  points  appeared  as  outliers  from  the  central 
cluster  of  points  in  the  factor  score  plot  of  Figure  5.2. 

A  somewhat  small  variability  in  the  center  point  replications  indicated  these 
outliers  were  distinct  from  the  central  region  of  points.  Flying  only  the  F-16  to  perform  its 
multi-role  missions  of  CAS,  air  superiority,  and  others  resulted  in  exposing  this  aircraft  to 
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a  multitude  of  dangerous  combat  situations.  Its  ability  to  perform  some  missions  better 
than  others  enters  into  the  picture,  since  the  achievement  of  mission  objectives  means 
using  every  capable  resource.  These  factors  affect  the  high  loss  of  aircraft  with  only  the 
F-16. 

The  low  loss  of  Blue  aircraft  was  seen  with  the  dominant  presence  of  air 
superiority  aircraft  and  jammers.  The  aircraft  of  these  missions  were  exposed  to  little  or 
no  low  altitude  attack;  thus  making  an  aircraft  loss  in  these  missions  due  to  air-to-air 
engagements.  These  losses  would  be  low,  due  to  the  superiority  of  allied  forces  pilots. 
Runs  with  these  aircraft  entering  the  force  mix  in  relatively  high  numbers,  compared  to  the 
rest,  composed  the  bottom  ridge  of  the  central  region  of  points. 

Along  the  upper  ridge  of  the  central  region  of  points,  those  runs  with  CAS- 
oriented  force  mixes  dominated.  Points  from  both  the  original  data  set  and  the  validation 
set  followed  this  pattern;  the  original  data  were  the  more  distinctive  cases.  The  Blue 
Attrition  Index  appeared  to  indicate  a  rank  order  of  force  mixes,  with  the  lower  scores 
corresponding  to  those  with  the  least  losses. 

Two  distinct  groups,  as  opposed  to  a  rank  order,  appeared  in  the  Air  Lethality 
Index.  The  factor  score  plot  found  in  Figure  5.3  clearly  shows  how  the  factor  scores  had 
separated  the  outputs  from  each  experimental  run  into  two  groups.  The  scores  along  the 
bottom  of  the  lower  group  matched  up  with  those  aircraft  force  mixes  more  oriented 
around  close  air  support-such  as  the  A-10,  F-16,  and  F-l  1 1;  the  F-15  did  not  appear 
among  these  force  mixes.  In  the  upper  group,  the  upper  ridge  of  the  group  was  indicative 
of  force  mixes  with  the  F-15  and  F-16— both  used  in  air  superiority  missions.  Taking  these 
indications  into  account,  it  appeared  that  the  group  with  negative  factor  scores  were  the 
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better  force  mixes  for  achieving  air  supremacy,  while  the  other  group  of  positive  scores 
was  not  as  effective  in  achieving  these  objectives. 


Validation  Factor  Scores: 

Ground  Objectives  Index  vs.  Air  Lethality  Index 


Figure  5.3.  Validation  Factor  Scores:  Ground  Objectives  Index 
versus  Air  Lethality  Index 
(Optimum:  Up,  Right) 


Summary 

The  use  of  a  validation  set  provided  some  credibility  for  the  results  obtained  from 
the  original  design.  The  additional  points  proved  most  useful  for  clarifying  the  principal 
components  analysis  ground  objectives  variable.  The  Ground  Objectives  Index  and  the 
Blue  Attrition  Index  indicated  the  same  optimum  force  mixes  as  previous.  The  Air 
lethality  index  again  indicated  two  different  groups  of  force  mixes.  The  addition  of  even 
more  runs  would  add  credibility  to  the  results  found  with  adding  the  validation  run 
outputs. 
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VI.  Summary,  Recommendations  for  Follow-On  Efforts,  and  Conclusions 


Summary 

The  research  of  this  thesis  investigated  the  effects  of  changing  force  structure  on 
THUNDER  output.  Modifications  of  the  force  structure  consisted  of  the  United  States 
Air  Force  primary  tactical  aircraft.  Each  aircraft  amount  was  limited  to  a  half  squadron 
(or  none  for  the  jammer  aircraft)  or  six  squadrons  (or  one  squadron  for  the  jammer 
aircraft).  These  different  limits  were  incorporated  into  an  experimental  design,  from  which 
a  response  surface  for  a  measure  of  effectiveness  was  generated.  The  metamodels  created 
were  found  to  be  adequate  to  predict  measures  of  effectiveness  within  the  range  of  each 
aircraft. 

Principal  components  analysis  and  factor  analysis  presented  a  relationship  among 
those  effectiveness  measures  relating  to  the  ground  war.  Achieving  the  ground  war 
objectives  most  efficiently  depended  upon  the  force  mix  used.  The  number  of  Blue 
aircraft  lost  appeared  to  be  minimized  with  the  increased  use  of  air  superiority  aircraft. 
Minimizing  the  number  of  days  to  achieve  air  supremacy  appeared  to  follow  suit,  as  two 
different  classes  of  force  mixes  emerged  from  the  analysis.  Those  measures  dealing  with 
the  air  war  were  weighted  towards  the  air  superiority  aircraft;  the  ground  war  was 
weighted  towards  the  CAS  aircraft.  Allocating  resources  towards  a  balance  of  this 
dichotomy  would  prove  essential  in  achieving  victory  in  a  large  scale  campaign. 
Recommendations  for  Follow-On  Efforts 

The  results  of  this  research  proved  most  interesting,  considering  the  size  of  the 
experiment,  number  of  variables,  and  number  of  outputs  observed.  The  research 
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presented  may  indicate  new  directions  for  finding  insight  as  to  how  THUNDER  models  a 
large-scale  campaign  and  interaction  among  inputs  and  outputs. 

An  unclassified  database  was  used  in  this  research  effort.  A  similar  research  effort 
using  a  classified  database  would  prove  useful  in  seeing  if  the  classified  parameters  change 
the  conclusions  from  this  research.  Performing  a  research  effort  like  the  one  presented  in 
this  thesis  with  a  classified  database  would  also  serve  as  another  validation  tool. 

The  EF-1 1 1 A  Raven,  at  the  time  of  this  writing,  is  scheduled  for  deactivation  by 
the  USAF  inventory.  The  experimental  design  observed  the  EF-1 1 1  A’s  presence  with  the 
absence  of  the  US  Navy’s  EA-6B  Prowler.  The  EF-1 1 1 A  proved  to  be  significant  in  all  of 
the  metamodels  produced.  An  interesting  investigation  would  be  to  deactivate  the  Ravens 
and  activate  all  the  Prowlers,  and  then  create  new  metamodels  to  see  how  the  Prowler  fits 
into  the  new  metamodels.  Then,  compare  the  different  metamodels  to  see  which  jammer 
had  the  greatest  influence.  A  fundamental  difference  between  the  Air  Force  and  Navy 
jammer  is  that  the  EF-1 1 1 A  is  supersonic;  the  EA-6B  is  not.  These  metamodel 
comparisons  could  provide  insight  to  the  importance  of  speed  in  an  airborne  jammer,  if 
that  were  a  factor  in  the  EF-1 1  lA’s  demise. 

Observing  the  validation  method  for  the  multivariate  analysis  indicates  more  data 
means  better  results  and  insight.  While  only  five  measures  of  effectiveness  and  a  grand 
total  of  32  runs  were  presented  in  the  multivariate  research,  more  interesting  conclusions 
could  be  made  with  twenty  or  more  MOEs  and  two  hundred  or  more  output  runs.  Such 
an  effort  would  demand  good  bookkeeping  to  document  the  parameters  of  each  run,  if 
they  are  not  part  of  an  experimental  design.  Knowing  the  dominant  factors  of  each  run 
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would  allow  the  runs  to  be  weighed  appropriately.  This  research  could  be  accomplished  in 
the  background  of  several  other  THUNDER  studies  over  a  period  of  months.  Using  a 
classified  database  would  bring  more  credibility  to  the  results;  unfortunately,  the  classified 
database  would  mean  a  classified  multivariate  analysis. 

Conclusion 

The  research  presented  in  this  thesis  has  shown  that  reasonably  good  metamodels 
can  be  produced  from  a  Resolution  V,  sixteen  run  experimental  design.  The  range  for 
number  of  each  aircraft  was  relatively  large;  however,  the  large  design  space  did  not  prove 
to  cause  the  creation  of  highly  irregular  response  surfaces.  Multivariate  Analysis  showed 
a  relationship  among  different  measures  of  effectiveness  and  the  potential  to  create  an 
“index”  for  use  in  quickly  deciphering  scenario  results.  This  multivariate  analysis,  along 
with  response  surface  methodology,  has  proved  to  be  extremely  useful  in  analyzing  the 
results  of  changing  force  structure  within  THUNDER. 
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APPENDIX  A 


The  following  is  the  text  to  the  squadron.dat  file  used  in  THUNDER.  Note  that 
the  only  values  changed  are  found  under  “AUTH .  QTY.” 


SQUADRONS. 3 05 

NUMBER . OF .MISSION . CLASSES :  9 

AIR. SUPERIORITY 
DEEP. STRIKE 
GROUND . SUPPORT 
JAMMER 
MULTI. ROLE 
RECCE 
WEASEL 
AWACS 
JSTARS 


NUMBER . OF . SORTIE . PROFILES :  1 7 

1001  "A- 10 " 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  4.00  5.00 

END. PROFILE 

1002  "F-16" 

DAY .  IN .  THEATER .  .  AUTH .  QTY .  SORT/DAY  .  .  AC  .  MAX .  SORT/DAY 
1.00  3.60  4.50 

6.00  2.50  3.50 

END . PROFILE 

1003  " RF-4 " 

DAY .  IN .  THEATER .  .  AUTH .  QTY .  SORT/DAY .  .  AC  .  MAX .  SORT/DAY 
1.00  2.50  3.00 

6.00  1.50  2.00 

END. PROFILE 

1004  "F-lll" 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  2.00  2.50 

6.00  1.20  1.50 

END . PROFILE 

1005  ” F-15” 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  3.00  3.50 

6.00  2.20  2.50 

END . PROFILE 

1006  "AV-8B" 

DAY .  IN .  THEATER .  .  AUTH  .  QTY .  SORT/ DAY  .  .AC  .MAX .  SORT/DAY 
1.00  4.00  5.00 


END . PROFILE 

1007  H  F/A- 18 " 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  3.60  4.00 

6.00  2.50  3.50 

END. PROFILE 

1008  "A-6" 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 


A-l 


1.00  2.00  2.50 

6.00  1.20  1.50 

END . PROFILE 

1009  " F-14  ” 

DAY.  IN. THEATER.  .  AUTH . QTY. SORT /DAY .  .AC  .MAX .SORT/DAY 
1.00  3.00  3.50 

6.00  2.20  2.50 

END . PROFILE 

1010  ”  E  -  3  " 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  .67  1.50 

END. PROFILE 

1011  "E-8" 

DAY .  IN .  THEATER .  .  AUTH .  QTY .  SORT/DAY .  .  AC  . MAX .  SORT/DAY 


1.00  .67  1.50 

END . PROFILE 

2001  "MIG-23” 

DAY .  IN .  THEATER .  .  AUTH .  QTY .  SORT/DAY .  .  AC  .  MAX .  SORT/DAY 
1.00  3.00  3.00 

6.00  1.20  1.20 

END. PROFILE 

2002  "MIRAGE  F-l” 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  4.00  4.00 

6.00  2.70  2.70 

END. PROFILE 

2003  "MIG-21” 

DAY .  IN .  THEATER .  .  AUTH .  QTY .  SORT/DAY .  .  AC .  MAX .  SORT/DAY 
1.00  3.00  3.00 

6.00  1.20  1.20 

END. PROFILE 

2004  "MIG-29” 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  4.00  4.00 

6.00  2.70  2.70 

END . PROFILE 

2005  " SU-25 " 

DAY .  IN .  THEATER .  .  AUTH .  QTY .  SORT/DAY .  .  AC  . MAX  .  SORT/DAY 
1.00  2.20  2.20 

6.00  .80  .80 

END. PROFILE 

2006  "MAINSTAY” 

DAY .  IN .  THEATER  .  .  AUTH .  QTY .  SORT/DAY  .  .  AC  .  MAX .  SORT/DAY 
1.00  .67  1.50 

END . PROFILE 


NUMBER . OF . SQUADRONS :  6 1 

11401  ” F14  USN  1" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH. QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1102  1009  20  1009  1009 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1013  1002  AIR. SUPERIORITY 

.  . DCA.  . ODCA.  .HVAA.  .BARC. .FSWP.  .  .RCA.  .  .STI. .  .CAS.  .  . BAI .  .  . INT .  .  .OCA 
100  100  100  100  100  000000 
.DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW . .SREC. .RESV 
0  0  0  0  0  0  0  100  0  0  0  100 
ORDERS 
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END . ORDERS 


10601  " A6E  USN  1" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1102  1008  25  1008  1008 

MOB. ID. . DISP . AB . ID . .MISSION. CLASS 
1013  1002  DEEP. STRIKE 

.  . DCA.  . ODCA .  . HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI. . .CAS.  .  . BAI .  .  . INT .  .  .OCA 
000000  100  100  100  100  100 
.DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW. .SREC. .RESV 
100  0000000000  100 
ORDERS 
END . ORDERS 


11801  "FA18  USN  1” 

SIDE.  .SUP. CMD. ID.  .TYPE.  AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1102  1007  20  1007  1007 

MOB. ID. . DISP . AB . ID . .MISSION . CLASS 
1013  1002  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP. . .RCA.  . .STI.  .  .CAS.  . .BAI.  . .INT.  .  .OCA 
100  100  100  100  100  0  100  100  100  100  100 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  100  0  0  0  100  0  0  0  100 

ORDERS 
END . ORDERS 

19601  " EA6B  USN  1" 

SIDE.  .SUP. CMD. ID. .TYPE.  AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1102  1008  0  1099  1008 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1013  1002  JAMMER 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA. . .STI.  .  .CAS.  .  .BAI.  .  .INT.  .  .OCA 
00000000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  100  100  100  0  0  0  0  100 
ORDERS 
END . ORDERS 

11402  " F14  USN  2" 

SIDE.  .SUP. CMD. ID.  .TYPE.  AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1102  1009  20  1009  1009 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1014  1002  AIR. SUPERIORITY 

.  .DCA.  .ODCA.  .HVAA.  .BARC. .FSWP.  .  .RCA.  .  .STI. . .CAS.  .  .BAI. . .INT. . .OCA 
100  100  100  100  100  000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  0  0  0  100  0  0  0  100 
ORDERS 
END . ORDERS 

10602  " A6E  USN  2" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1102  1008  25  1008  1008 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1014  1002  DEEP. STRIKE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  . .RCA.  .  .STI.  .  .CAS.  . .BAI.  .  .INT.  .  .OCA 
000000  100  100  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
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100  0000000000  100 
ORDERS 
END . ORDERS 

11802  11 FA18  USN  2" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1102  1007  20  1007  1007 

MOB. ID.  . DISP .  AB  . ID .  .MISSION. CLASS 
1014  1002  MULTI. ROLE 

.  . DCA.  . ODCA .  . HVAA.  .BARC. .FSWP.  .  .RCA.  .  .STI.  . .CAS.  .  .BAI . .  . INT .  . .OCA 
100  100  100  100  100  0  100  100  100  100  100 

. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW. .SREC. .RESV 
100  100  100  100  000  100  000  100 

ORDERS 
END . ORDERS 

19602  " EA6B  USN  2” 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1102  1008  0  1099  1008 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 

1014  1002  JAMMER 

.  .DCA. .ODCA.  .HVAA.  .BARC. .FSWP.  .  .RCA.  . .STI. . .CAS.  .  .BAI.  .  .INT. .  .OCA 
00000000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  100  100  100  0  0  0  0  100 
ORDERS 
END . ORDERS 

11403  " F14  USN  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1103  1009  20  1009  1009 

MOB. ID. .DISP.AB.ID. . MISSION . CLASS 

1015  1002  AIR. SUPERIORITY 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

.  .CAS. 

.  .BAI.  , 

.  .INT. 

.  .OCA 

100 

100 

100 

100 

100 

0 

0 

0 

0 

0 

0 

DSED. 

.SSUP. 

.  CSUP . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC. , 

.  .AEW. 

.  SREC . 

.RESV 

0 

0 

0 

0 

0 

0 

0 

100 

0 

0 

0 

100 

ORDERS 
END . ORDERS 


10603  H A6E  USN  3” 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1103  1008  25  1008  1008 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1015  1002  DEEP. STRIKE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS. .  .BAI.  .  .INT.  .  .OCA 
000000  100  100  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  0000000000  100 
ORDERS 
END . ORDERS 

11803  " FA1 8  USN  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1103  1007  20  1007  1007 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1015  1002  MULTI. ROLE 

. .DCA.  .ODCA.  .HVAA. .BARC.  .FSWP.  .  .RCA.  . .STI.  .  .CAS.  . .BAI. . .INT. . .OCA 
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100  100  100  100  100  0  100  100  100  100  100 

.DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW. . SREC . .RESV 
100  100  100  100  000  100  000  100 

ORDERS 
END . ORDERS 

19603  " EA6B  USN  3” 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. .SERVE . KIT . ID . . SORT . PROF . ID 
1  1103  1008  0  1099  1008 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1015  1002  JAMMER 

.  .DCA.  . ODCA .  .HVAA.  .BARC.  . FSWP .  . .RCA.  .  .STI.  .  .CAS.  .  .BAI . .  . INT .  .  .OCA 
00000000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  100  100  100  0  0  0  0  100 
ORDERS 
END . ORDERS 

10604  " A6E  USMC  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1102  1008  25  1008  1008 

MOB . ID . . DISP . AB . ID . . MISSION . CLASS 
1002  1017  DEEP. STRIKE 

.  .DCA.  .ODCA.  .HVAA.  .BARC. .FSWP.  .  .RCA.  .  .STI. . .CAS.  .  .BAI. . .INT.  .  .OCA 
000000  100  100  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  0000000000  100 
ORDERS 
END . ORDERS 

11804  " FA1 8  USMC  1" 

SIDE.  .SUP. CMD. ID. .TYPE.  AC. ID.  .AUTH.QTY. . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1102  1007  48  1007  1007 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1002  1017  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS. .  .BAI.  .  .INT.  .  .OCA 
100  100  100  100  100  0  100  100  100  100  100 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  100  0  0  0  100  0  0  0  100 

ORDERS 
END . ORDERS 

10804  "  AV8B  USMC  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1102  1006  60  1006  1006 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1002  1017  GROUND . SUPPORT 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 
000000  100  100  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  0000000000  100 
ORDERS 
END . ORDERS 

11001  nA10  1" 

SIDE.  .SUP. CMD. ID.  .TYPE.  AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1104  1001  26  1001  1001 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 


A-5 


1010  1022  GROUND . SUPPORT 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

.  FSWP  . 

.  .RCA. 

.  .STI. 

. .CAS. 

.  .BAI.  , 

.  .INT. 

.  .OCA 

0 

0 

0 

0 

0 

0 

100 

100 

100 

100 

100 

DSED. 

.  SSUP . 

.  CSUP . 

. ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC.  , 

.  .AEW. 

.SREC. 

.RESV 

100 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

100 

ORDERS 
END . ORDERS 


11002  "A10  2" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1001  26  1001  1001 
MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1025  1022  GROUND. SUPPORT 

.  .DCA.  . ODCA .  . HVAA.  .BARC.  . FSWP .  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI.  .  . INT .  .  .OCA 
000000  100  100  100  100  100 
. DSED . .SSUP. . CSUP . .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW . .SREC. .RESV 
100  0000000000  100 
ORDERS 
END . ORDERS 

11003  ,f A10  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . .SORT . PROF. ID 
1  1104  1001  26  1001  1001 
MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1022  1010  GROUND . SUPPORT 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI. .  .CAS.  .  .BAI.  .  .INT.  .  .OCA 
000000  100  100  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  0000000000  100 
ORDERS 
END . ORDERS 

11601  ” FI 6  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1002  29  1002  1002 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1026  1022  MULTI. ROLE 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

.  .CAS. 

.  .BAI. 

.  . INT . 

.  .OCA 

100 

100 

100 

100 

100 

0 

100 

50 

50 

100 

100 

DSED. 

.  SSUP . 

.  CSUP . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.  EAIR . 

.RECC.  . 

.  .AEW. 

.  SREC . 

.RESV 

50 

0 

0 

0 

0 

0 

0 

100 

0 

0 

0 

100 

ORDERS 
END . ORDERS 

11602  “ FI 6  2” 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1002  29  1002  1002 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1022  1026  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP. . .RCA.  .  .STI. .  .CAS.  .  .BAI.  .  .INT.  .  .OCA 
100  100  100  100  100  0  100  100  100  100  100 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
50  0  0  0  0  0  0  100  0  0  0  100 

ORDERS 
END . ORDERS 

11603  " FI 6  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
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1  1104  1002  29  1002  1002 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1005  1026  MCJLTI  .ROLE 

. . DCA. . ODCA. . HVAA. .BARC. . FSWP . . .RCA. . .STI. . .CAS. . . BAI . . . INT . . .OCA 
100  100  100  100  100  0  100  10  10  100  100 

. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW . .SREC. .RESV 
50  0  0  0  0  0  0  100  0  0  0  100 

ORDERS 
END . ORDERS 

11604  "WEASEL  1" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1002  24  1002  1002 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1026  1022  WEASEL 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

.  .CAS. 

.  .BAI.  , 

.  .INT. 

.  .OCA 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

DSED. 

.SSUP. 

.  CSUP . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC.  , 

.  .AEW. 

.  SREC . 

.RESV 

100 

100 

100 

100 

0 

0 

0 

0 

0 

0 

0 

100 

ORDERS 
END . ORDERS 


10401  "RF4  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1003  24  1003  1003 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1023  1022  RECCE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  . .STI.  .  .CAS.  .  .BAI.  . .INT.  .  .OCA 
00000000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
00000000  100  0  0  100 
ORDERS 
END . ORDERS 

10301  "AWACS" 


SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 


1  1104 

1010 

12 

1003 

1010 

MOB. ID. .DISP 

.  AB . ID . 

.  .MISSION. CLASS 

1001 

1020 

AWACS 

.  .DCA.  .ODCA. 

.HVAA. . 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

.  .CAS. 

.  .BAI.  . 

.  . INT .  , 

.  .OCA 

0  0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

.DSED. .SSUP. 

.  CSUP .  . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC. , 

.  .AEW.  , 

.  SREC . 

.RESV 

0  0 

0 

0 

0 

0 

0 

0 

0 

100 

0 

100 

ORDERS 

END . ORDERS 

10801  " JSTARS ” 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1011  6  1003  1011 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1001  1020  JSTARS 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  . .BAI.  .  .INT.  .  .OCA 
00000000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0000000000  100  100 
ORDERS 
END . ORDERS 
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11101  "Fill  1" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1004  39  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1029  1027  DEEP  ..STRIKE 

.  . DCA.  . ODCA .  .HVAA .  .BARC.  . FSWP .  . .RCA.  .  .STI.  .  .CAS. .  . BAI .  .  . INT .  .  .OCA 
0  0  0  0  0  0  100  10  100  100  100 
. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW. .SREC. .RESV 
50  0000000000  100 

ORDERS 
END . ORDERS 

11102  "Fill  2" 

SIDE.  .SUP. CMD. ID.  .TYPE.  AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1104  1004  39  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1027  1029  DEEP. STRIKE 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

. FSWP . 

.  .RCA. 

. .STI. 

.  .CAS. 

.  .BAI. 

.  .INT. 

.  .OCA 

0 

0 

0 

0 

0 

0 

100 

10 

100 

100 

100 

DSED. 

.  SSUP . 

.  CSUP . 

. ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC. 

.  .AEW. 

.  SREC . 

.RESV 

50 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

100 

ORDERS 
END . ORDERS 


11103  " F15E  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1004  0  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1039  1007  DEEP. STRIKE 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 
0  0  0  0  0  0  100  10  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
50  0000000000  100 

ORDERS 
END . ORDERS 

11104  " F117  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1004  0  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1007  1004  DEEP. STRIKE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA. . .STI. . .CAS.  .  .BAI.  .  .INT.  .  .OCA 
0  0  0  0  0  0  100  0  0  100  50 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
50  0000000000  100 

ORDERS 
END . ORDERS 

11105  "TORN  IDS  1" 

SIDE.  .SUP. CMD. ID.  .TYPE.  AC. ID.  .AUTH.QTY. . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1104  1004  0  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1016  1018  DEEP. STRIKE 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

. .CAS, 

.  .BAI. 

.  .INT. 

.  .OCA 

0 

0 

0 

0 

0 

0 

100 

100 

100 

100 

100 

DSED. 

.  SSUP . 

.  CSUP . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC. 

.  .AEW. 

.SREC. 

.RESV 

50 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

100 

ORDERS 
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END . ORDERS 


11106  ’’TORN  IDS  2” 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1004  0  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
1018  1016  DEEP. STRIKE 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

. .CAS. 

.  .BAI.  , 

.  .INT. 

.  .OCA 

0 

0 

0 

0 

0 

0 

100 

100 

100 

100 

100 

DSED. 

.SSUP. 

.  CSUP . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC.  , 

.  .AEW. 

.  SREC . 

.RESV 

50 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

100 

ORDERS 
END . ORDERS 


11107  "TORN  IDS  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1004  0  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1021  1016  DEEP. STRIKE 

.  .DCA.  . ODCA .  . HVAA.  .BARC.  . FSWP .  .  .RCA.  . .STI. .  .CAS.  .  .BAI  .  .  . INT .  .  .OCA 
000000  100  100  100  100  100 
. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . .AEW. .SREC. .RESV 
50  0000000000  100 
ORDERS 
END . ORDERS 

11108  ’’TORN  IDS  4" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1004  0  1004  1004 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1016  1018  DEEP. STRIKE 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 
000000  100  100  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
50  0000000000  100 

ORDERS 
END . ORDERS 

11109  ” EF111  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1004  12  1099  1004 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1004  1007  JAMMER 

. .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  . .RCA.  .  .STI.  .  .CAS.  . .BAI.  .  .INT.  . .OCA 
00000000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  100  100  100  0  0  0  0  100 
ORDERS 
END . ORDERS 

11501  " FI 5  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1005  13  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
1021  1023  AIR. SUPERIORITY 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 
100  100  100  100  100  000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
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0  0  0  0  0  0  0  100  0  0  0  100 
ORDERS 
END . ORDERS 

11502  M FI 5  2" 

SIDE.  . SUP . CMD  . ID .  .TYPE. AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1104  1005  13  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1023  1021  AIR. SUPERIORITY 

.  . DCA.  . ODCA .  .HVAA.  .BARC. .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  .  . BAI  .  .  . I NT .  .  .OCA 
100  100  100  100  100  000000 
. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . .AEW . .SREC. .RESV 
0  0  0  0  0  0  0  100  0  0  0  100 
ORDERS 
END . ORDERS 

11503  " F15  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1005  13  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 

1016  1022  AIR. SUPERIORITY 


.DCA. 

.ODCA. 

.HVAA. 

.BARC. 

.  FSWP . 

.  .RCA. 

.  .STI. 

. .CAS. 

.  .BAI. 

.  .INT. 

.  .OCA 

100 

100 

100 

100 

100 

0 

0 

0 

0 

0 

0 

DSED. 

.  SSUP . 

. CSUP . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC. 

.  .AEW. 

.SREC. 

.RESV 

0 

0 

0 

0 

0 

0 

0 

100 

0 

0 

0 

100 

ORDERS 
END . ORDERS 


11504  " FI 5  4" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1005  13  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1022  1016  AIR. SUPERIORITY 

.  .DCA.  .ODCA.  .HVAA.  .BARC. .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI.  .  . I NT .  .  .OCA 
100  100  100  100  100  000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  0  0  0  100  0  0  0  100 
ORDERS 
END . ORDERS 

11505  "FI 5  5" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . .SORT. PROF. ID 
1  1104  1005  13  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1029  1030  AIR. SUPERIORITY 

.  .DCA.  .ODCA. .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI.  .  . INT .  .  .OCA 
100  100  100  100  100  000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  0  0  0  100  0  0  0  100 
ORDERS 
END . ORDERS 

11506  " FI 5  6" 

SIDE.  .SUP. CMD. ID.  .TYPE. AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
1  1104  1005  13  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1030  1029  AIR. SUPERIORITY 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 


A-10 


100  100  100  100  100  000000 
. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. . EJAM. . EAIR . .RECC. . . AEW. .SREC. .RESV 
0000000  100  000  100 

ORDERS 
END .  ORDERS 

11507  ’’TORN  ADV  1" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. .SERVE . KIT . ID . . SORT . PROF . ID 
1  1104  1005  0  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 

1018  1024  AIR. SUPERIORITY 

.  . DCA.  . ODCA.  . HVAA.  .BARC.  . FSWP .  .  .RCA.  .  .STI. .  .CAS.  .  . BAI .  .  . INT .  .  .OCA 
100  100  100  100  100  0  0  0  0  0  0 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
0  0  0  0  0  0  0  100  0  0  0  100 

ORDERS 
END . ORDERS 

11508  M TORN  ADV  2" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 

1  1104  1005  0  1005  1005 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

1024  1018  AIR. SUPERIORITY 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI.  .  .INT.  .  .OCA 
100  100  100  100  0  0  0  0  0  0  0 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
00000000000  100 
ORDERS 
END . ORDERS 

22901  "MIG29  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 

2  2101  2004  38  2004  2004 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

2010  2013  AIR. SUPERIORITY 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 
100  100  100  100  100  000000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
00000000000  100 
ORDERS 
END . ORDERS 

22101  "MIG21  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2003  38  2003  2003 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
2013  2010  AIR. SUPERIORITY 

.  . DCA .  . ODCA .  . HVAA .  . BARC . . FSWP . . . RCA. . . STI .  .  . CAS . . . BAI  .  .  . INT .  .  . OCA 
100  00  10  000  10  000 
.DSED.  .SSUP.  .CSUP.  .ESUP.  .SJAM.  .CJAM.  .EJAM.  .EAIR.  .RECC.  .  .AEW.  .SREC.  .RESV 
00000000000  100 
ORDERS 
END . ORDERS 

22102  "MIG21  2" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2003  38  2003  2003 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
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2029  2032  AIR . SUPERIORITY 

.  . DCA.  . ODCA.  . HVAA.  .BARC. . FSWP .  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI .  .  .INT.  .  .OCA 
100  00  10  000  10  000 
.DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR . .RECC. . . AEW. .SREC. .RESV 
0  0  000000000  100 
ORDERS 
END . ORDERS 

22103  "MIG21  3" 

SIDE.  . SUP . CMD . ID .  .TYPE. AC. ID.  .AUTH.QTY.  . SERVE  . KIT . ID .  . SORT . PROF . ID 
2  2101  2003  38  2003  2003 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

2032  2029  AIR . SUPERIORITY 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI.  .  .INT.  .  .OCA 
100  0  0  10  0  0  0  10  0  0  0 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
00000000000  100 
ORDERS 
END . ORDERS 

22104  "MIG21  4” 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . .SORT . PROF. ID 
2  2101  2003  38  2003  2003 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 

2004  2009  AIR. SUPERIORITY 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 
100  00  10  000  10  000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
00000000000  100 
ORDERS 
END . ORDERS 

22105  "MIG21  5" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2003  38  2003  2003 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

2019  2020  AIR. SUPERIORITY 

.  .DCA.  .ODCA.  .HVAA.  .BARC. .FSWP.  .  .RCA. . .STI. . .CAS.  .  .BAI.  .  .INT.  . .OCA 
100  00  10  000  10  000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
00000000000  100 
ORDERS 
END . ORDERS 

22106  "MIG21  6" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2003  38  2003  2003 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

2020  2019  AIR. SUPERIORITY 

. .DCA. .ODCA. .HVAA. .BARC. .FSWP. . .RCA. . .STI. . .CAS. . .BAI. . .INT. . .OCA 
100  00  10  000  10  000 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
00000000000  100 
ORDERS 
END . ORDERS 

20101  "MIRAGE  FI  1" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
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2  2101  2002  38  2002  2002 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
2023  2022  MULTI. ROLE 

.  . DCA.  . ODCA.  . HVAA.  .BARC.  . FSWP .  . .RCA.  .  .STI.  .  .CAS.  . .BAI.  .  . INT .  .  .OCA 
100  100  100  100  100  0  10  0  0  10  10 

. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW . .SREC. .RESV 
100  100  100  10  0000000  100 
ORDERS 
END . ORDERS 

20102  "MIRAGE  FI  2" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2002  38  2002  2002 

MOB . ID . . DISP . AB . ID . . MISSION . CLASS 
2019  2010  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  . .STI. . .CAS.  .  .BAI. .  .INT.  .  .OCA 
100  100  100  100  100  0  10  0  0  10  10 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  10  0000000  100 

ORDERS 
END . ORDERS 

20103  "MIRAGE  FI  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2002  38  2002  2002 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 

2007  2006  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  . .RCA. . .STI .  .  .CAS.  . .BAI .  .  .INT.  .  .OCA 
100  100  100  100  100  000000 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  100  000  100  0  0  .  0  100 

ORDERS 
END . ORDERS 

20104  "MIRAGE  FI  4" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2002  38  2002  2002 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
2006  2008  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC. .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI.  .  .INT.  .  .OCA 
100  100  100  100  100  0  100  100  100  100  100 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  100  000  100  000  100 

ORDERS 
END . ORDERS 

20105  "MIRAGE  FI  5" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2002  38  2002  2002 

MOB . ID . . DISP . AB . ID .. MISSION . CLASS 

2008  2007  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  .  .BAI.  .  .INT.  .  .OCA 
100  100  100  100  100  0  50  100  100  100  100 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  100  0  0  0  0  0  0  0  100 

ORDERS 
END . ORDERS 

i 
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22501  " SU25  1" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. .SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2005  38  2005  2005 

MOB. ID. .DISP.AB.ID. .MISSION. CLASS 
2001  2005  GROUND. SUP PORT 

.  . DCA.  . ODCA .  .HVAA.  .BARC.  .FSWP. . .RCA. . .STI. . .CAS.  .  . BAI .  .  . INT .  .  .OCA 
0  0  0  0  0  0  0  100  100  0  0 

. DSED . .SS UP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . .AEW . .SREC. .RESV 
000000000000 
ORDERS 
END . ORDERS 

22502  " SU25  2" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2005  38  2005  2005 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
2005  2001  GROUND . SUPPORT 

.  .DCA.  .ODCA.  .HVAA.  .BARC. . FSWP .  .  .RCA. . .STI. .  .CAS. . .BAI.  . .INT.  .  .OCA 
0  0  0  0  0  0  0  100  100  0  0 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
000000000000 
ORDERS 
END . ORDERS 

22301  "MIG23  1” 

SIDE.  .SUP. CMD. ID.  .TYPE.  AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID .  . SORT . PROF . ID 
2  2101  2001  38  2001  2001 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
2003  2004  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI. .  .CAS.  . .BAI.  .  .INT.  .  .OCA 
100  100  100  100  100  0  100  100  100  100  100 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  100  000  100  000  100 

ORDERS 
END . ORDERS 

22302  "MIG23  2" 

SIDE.  .SUP. CMD. ID.  .TYPE.  AC. ID.  .AUTH.QTY.  . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2001  38  2001  2001 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 


2011 

2012 

MULTI .ROLE 

.  .DCA.  , 

.ODCA. 

.HVAA. 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

.  .CAS. 

.  .BAI  . 

.  .INT. 

.  .OCA 

100 

100 

100 

100 

100 

0 

100 

100 

100 

100 

100 

.DSED. . 

.  SSUP . 

. CSUP . 

.  ESUP . 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC. 

.  .AEW. 

.SREC. 

.RESV 

100 

ORDERS 

100 

100 

100 

0 

0 

0 

100 

0 

0 

0 

100 

END . ORDERS 

22303  "MIG23  3" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 
2  2101  2001  38  2001  2001 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
2018  2025  MULTI. ROLE 

.  .DCA.  .ODCA.  .HVAA.  .BARC.  .FSWP.  .  .RCA.  .  .STI.  .  .CAS.  . .BAI.  .  .INT.  .  .OCA 
100  100  100  100  100  0  100  100  100  100  100 

.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .EAIR. .RECC. . .AEW. .SREC. .RESV 
100  100  100  100  000  100  000  100 

ORDERS 
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END . ORDERS 


22304  "MIG23  4" 

SIDE. . SUP . CMD . ID . .TYPE. AC. ID. .AUTH.QTY. .SERVE .KIT . ID . . SORT . PROF . ID 
2  2101  2001  38  2001  2001 

MOB. ID. .DISP.AB.ID. .MISSION . CLASS 
2021  2024  MULTI. ROLE 

.  . DCA.  . ODCA.  . HVAA.  .BARC.  . FSWP .  .  .RCA. . .STI.  .  .CAS.  .  .BAI .  .  . INT .  . .OCA 
100  100  100  100  100  0  100  100  100  100  100 

. DSED . .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. . EAIR . .RECC. . . AEW. . SREC . .RESV 
100  100  100  100  000  100  000  100 

ORDERS 
END . ORDERS 


20000  "MAINSTAY" 

SIDE. .SUP. CMD. ID. .TYPE. AC. ID. .AUTH.QTY. . SERVE . KIT . ID . . SORT . PROF . ID 


2  2101 

MOB. ID. .DISP.AB.ID, 

2014  2013 

2006  6 

.  .MISSION. CLASS 

AWACS 

2006 

2006 

.  .DCA.  .ODCA. 

.HVAA.  , 

.BARC. 

. FSWP . 

.  .RCA. 

.  .STI. 

. . CAS . 

.  .BAI. 

.  .  INT . 

.  .OCA 

0  0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

.DSED. .SSUP. 

.  CSUP .  . 

.ESUP. 

.SJAM. 

.CJAM. 

.EJAM. 

.EAIR. 

.RECC. 

.  .AEW. 

.SREC. 

.RESV 

0  0 

ORDERS 

END . ORDERS 

0 

0 

0 

0 

0 

0 

0 

100 

0 

100 

END. SQUADRONS 
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APPENDIX  C 


The  following  tables  are  the  complete  results  from  performing  regression  analysis 
with  only  the  most  significant  (90%  and  above)  terms  included.  Residual  plots  for  each 
regression  are  also  presented. 

STATISTIX  4.1 

UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  OF  DAYS  TO  NEUTRALIZE  FLOT 
PREDICTOR 


VARIABLES 

COEFFICIENT 

STD  ERROR 

STUDENT ' S 

T  P 

VI F 

CONSTANT 

13.5000 

0.06250 

216.00 

0.0000 

A10 

-0 . 56250 

0.06250 

-9.00 

0.0000 

1.0 

F15 

-0.12500 

0.06250 

-2.00 

0.0805 

1.0 

FI  6 

-0.56250 

0.06250 

-9.00 

0.0000 

1.0 

EF111 

-0.12500 

0.06250 

-2.00 

0.0805 

1.0 

A10F15 

0.18750 

0.06250 

3.00 

0.0171 

1.0 

A10EF 

0.31250 

0.06250 

5.00 

0.0011 

1 . 0 

F15F111 

-0.12500 

0.06250 

-2.00 

0.0805 

1 . 0 

R-SQUARED 

0.9630 

RESIDUAL 

MEAN  SQUARE 

(MSE)  0, 

.06250 

ADJUSTED  R-, 

SQUARED  0.9306 

STANDARD 

ERROR  OF  ESTIMATE  0, 

.25000 

SOURCE 

DF  SS 

MS 

F 

P 

REGRESSION 

7  13.0000 

1.85714  29.71 

0.0000 

RESIDUAL 

8  0.50000 

0.06250 

TOTAL 

15  13.5000 

CASES  INCLUDED  16  MISSING 

CASES  0 

Residual  Plot  of  Days  to  Neut 

FLOT 

12.1  12.8  11.5  14.2  14.9  15.6 

Predicted 
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STATISTIX  4.1 


UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  OF  DAYS  TO  AIR  SUPREMACY 
PREDICTOR 


VARIABLES 

COEFFICIENT 

STD  ERROR 

STUDENT'S  T 

P  VIF 

CONSTANT 

16.6250 

0.27003 

61.57 

0.0000 

FI  5 

-1.25000 

0.27003 

-4.63 

0.0012  1.0 

F16 

-0.50000 

0.27003 

-1.85 

0.0971  1.0 

Fill 

-0.50000 

0.27003 

-1.85 

0.0971  1.0 

EF111 

0.75000 

0.27003 

2.78 

0.0215  1.0 

A10F111 

-0.75000 

0.27003 

-2.78 

0.0215  1.0 

F16EF 

0.87500 

0.27003 

3.24 

0.0102  1.0 

R-SQUARED 

0.8576 

RESIDUAL 

MEAN  SQUARE 

(MSE)  1.16667 

ADJUSTED  R- 

■SQUARED  0.7627 

STANDARD 

ERROR  OF  ESTIMATE  1.08012 

SOURCE 

DF  SS 

MS 

F 

P 

REGRESSION 

RESIDUAL 

TOTAL 

6  63.2500 

9  10.5000 

15  73.7500 

10.5417 

1.16667 

9.04 

0.0022 

CASES  INCLUDED  16  MISSING  CASES  0 


Residual  Plot  of  Days  to  Air  Supremacy 


Predicted 


STATISTIX  4.1 


UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  OF  NUMBER  OF  AIR  KILLS 
PREDICTOR 


VARIABLES 

COEFFICIENT  1 

STD  ERROR 

STUDENT'S  T 

P 

VIF 

CONSTANT 

9215.69 

57.3476 

160.70 

0.0000 

A10 

1622.19 

57.3476 

28.29 

0.0000 

1.0 

F16 

2058.06 

57.3476 

35.89 

0.0000 

1.0 

Fill 

256.063 

57.3476 

4.47 

0.0029 

1.0 

EF111 

147.937 

57.3476 

2.58 

0.0365 

1.0 

A10EF 

-294.563 

57.3476 

-5.14 

0.0013 

1.0 

F15F16 

-176.188 

57.3476 

-3.07 

0.0180 

1.0 

F15F111 

-141.937 

57.3476 

-2.48 

0.0425 

1.0 

F16EF 

-183 . 688 

57.3476 

-3.20 

0.0150 

1.0 

R- SQUARED 

0.9968 

RESIDUAL 

MEAN  SQUARE 

(MSE)  52619 

.9 

ADJUSTED  R- 

-SQUARED  0.9931 

STANDARD 

ERROR  OF  ESTIMATE  229.390 

SOURCE 

DF  SS 

MS 

F 

P 

REGRESSION 

8  1 . 140E+08 

1 . 425E+07 

270.86 

0.0000 

RESIDUAL 

7  3 . 683E+05 

52619.9 

TOTAL 

15  1 . 144E+08 

CASES  INCLUDED  16  MISSING  CASES  0 


Residual  Plot  of  Number  of  Air  Kills 


C-3 


STATISTIX  4.1 


UNWEIGHTED 

PREDICTOR 

VARIABLES 

LEAST  SQUARES  LINEAR  REGRESSION  OF  BLUE  AIRCRAFT  LOST 

COEFFICIENT  STD  ERROR  STUDENT'S  T  P 

VIF 

CONSTANT 

115.438 

1.16089 

99.44 

0.0000 

A10 

9.68750 

1.16089 

8.34 

0.0004 

1.0 

FI  5 

-4.68750 

1.16089 

-4.04 

0.0099 

1.0 

F16 

8.81250 

1.16089 

7.59 

0.0006 

1.0 

EF111 

-3.68750 

1.16089 

-3.18 

0.0246 

1.0 

A10F15 

3.06250 

1.16089 

2.64 

0.0461 

1.0 

A10F111 

4.06250 

1.16089 

3.50 

0.0173 

1.0 

A10EF 

5.31250 

1.16089 

4.58 

0.0060 

1 . 0 

F15F16 

-4.81250 

1.16089 

-4.15 

0.0089 

1.0 

F15F111 

6.43750 

1.16089 

5.55 

0.0026 

1.0 

F111EF 

5.68750 

1.16089 

4.90 

0.0045 

1.0 

R-SQUARED 

0.9815 

RESIDUAL 

MEAN  SQUARE 

(MSE)  21 

.5625 

ADJUSTED  R- 

-SQUARED  0.9446 

STANDARD 

ERROR  OF  ESTIMATE  4. 

64354 

SOURCE 

DF  SS 

MS 

F 

P 

REGRESSION 

10  5730.13 

573.013 

26.57 

0.0010 

RESIDUAL 

5  107.812 

21.5625 

TOTAL 

15  5837.94 

CASES  INCLUDED  16  MISSING 

CASES  0 

Residual  Plot  for  Blue  Aircraft  Lost 


60  100  120  140 


Predicted 


STATISTIX  4.1 


UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  OF  DEPTH  OF  ENEMY  ADVANCE 
PREDICTOR 


VARIABLES 

COEFFICIENT  STD  ERROR 

STUDENT'S  T 

P 

VI F 

CONSTANT 

56.6250 

0.28868 

196.15 

0.0000 

A10 

-2.00000 

0.28868 

-6.93 

0.0001 

1.0 

FI  5 

-0.87500 

0.28868 

-3.03 

0.0142 

1.0 

FI  6 

-4.12500 

0.28868 

-14.29 

0 . 0000 

1.0 

EF111 

-1.00000 

0.28868 

-3.46 

0.0071 

1.0 

F15EF 

-0.75000 

0.28868 

-2.60 

0.0288 

1.0 

A10EF 

0.62500 

0.28868 

2.17 

0.0586 

1 . 0 

R- SQUARED 

0.9694 

RESIDUAL  : 

MEAN  SQUARE 

(MSE)  1.33333 

ADJUSTED  R- 

SQUARED  0.9489 

STANDARD 

ERROR  OF  ESTIMATE  1.15470 

SOURCE 

DF  SS 

MS 

F 

P 

REGRESSION 

6  379.750 

63.2917 

47.47 

0.0000 

RESIDUAL 

9  12.0000 

1.33333 

TOTAL 

15  391.750 

CASES  INCLUDED  16  MISSING  CASES  0 


Residual  Plot  of  Depth  of  Enemy  Advance 


47  53  59  55 

Predicted 
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APPENDIX  D 


The  following  tables  and  plots  are  the  quadratic  polynomial  regression  results  for 
MOE  #3,  Air  Kills,  and  MOE  #5,  Depth  of  Enemy  Advance— keeping  only  those  terms 
having  a  90%  level  of  significance. 


STATISTIX  4.1 

UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  OF  NUMBER  OF  AIR  KILLS 
PREDICTOR 


VARIABLES 

COEFFICIENT 

STD  ERROR 

STUDENT 1 S 

T  P 

VI F 

CONSTANT 

9015.45 

74.7087 

120.67 

0.0000 

A10 

1605.94 

58.4025 

27.50 

0.0000 

1 . 0 

F16 

2059.94 

58.4025 

35.27 

0.0000 

1.0 

Fill 

238.833 

58.4025 

4.09 

0.0006 

1.0 

EF111 

107.167 

58.4025 

1.83 

0.0822 

1.0 

A10EF 

-294.563 

61.9452 

-4.76 

0.0001 

1.0 

F15F16 

-176.188 

61.9452 

-2.84 

0.0104 

1.0 

F15F111 

-141.937 

61.9452 

-2.29 

0.0335 

1 . 0 

F16EF 

-183.688 

61.9452 

-2.97 

0.0079 

1.0 

F15F15 

229.157 

94.8274 

2.42 

0.0259 

1.0 

R- SQUARED 

0.9909 

RESIDUAL 

MEAN  SQUARE 

(MSE)  61395.3 

ADJUSTED  R- 

-SQUARED  0.9866 

STANDARD 

ERROR  OF  ESTIMATE  247. 

.781 

SOURCE 

DF  SS 

MS 

F 

P 

REGRESSION 

9  1.271E+08 

1 .413E+07 

230.10 

0.0000 

RESIDUAL 

19  1 . 167E+06 

61395.3 

TOTAL 

28  1.283E+08 

CASES  INCLUDED  29  MISSING  < 

CASES  0 

D-l 


Residual  Plot  of  Air  Kills  (Quadratic) 


Residual  Plot  of  Depth  of  Advance  (Quad) 


53  59  65 


Predicted 


STATISTIX  4.1 


UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  OF  DEPTH  OF  ENEMY  ADVANCE 
PREDICTOR 


VARIABLES 

COEFFICIENT  STD  ERROR 

STUDENT'S  T 

P 

VI F 

CONSTANT 

56.3905 

0.49290 

114.41 

0.0000 

A10 

-2.00000 

0.36630 

-5.46 

0.0000 

1.0 

F15 

-0.66667 

0.36630 

-1.82 

0.0838 

1.0 

F16 

-4.27778 

0.36630 

-11.68 

0.0000 

1 . 0 

Fill 

-0.66667 

0.36630 

-1.82 

0.0838 

1.0 

EF111 

-0.94444 

0.36630 

i 

to 

in 

00 

0.0179 

1.0 

F15EF 

-0.75000 

0.38852 

-1.93 

0.0679 

1.0 

F15F15 

-2.14793 

0.84108 

-2.55 

0.0189 

2.0 

F16F16 

2.35207 

0.84108 

to 

oo 

o 

0.0111 

2.0 

R- SQUARED 

0.9055 

RESIDUAL 

MEAN  SQUARE  1 

(MSE)  2.41520 

ADJUSTED  R-, 

SQUARED  0.8677 

STANDARD 

ERROR  OF  ESTIMATE  1.55409 

SOURCE 

DF  SS 

MS 

F 

P 

REGRESSION 

8  462.937 

57.8672  23.96  C 

)  .0000 

RESIDUAL 

20  48.3041 

2.41520 

TOTAL 

28  511.241 

CASES  INCLUDED  29  MISSING  CASES  0 
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APPENDIX  E 


Principal  Components  Analysis  using  Mathcad  PLUS  5.0 


i  :=  0..  15 


14.5 

19 

35522 

91 

64 

COLUMN  1: 

1 

13 

19 

33683 

120 

58 

Days  Needed  to  Neutralize  FLOT 

1 

15 

16 

35829 

101 

65 

1 

14 

18 

33405 

106 

58 

COLUMN  2: 

Days  to  Air  Supremacy 

1 

14 

15 

34078 

155 

56 

1 

12.5 

19 

32288 

135 

52 

COLUMN  3: 

1 

13 

17 

33964 

87 

52 

Red  Inventory  Remaining  at  War's  end 

1 

12 

14 

31854 

118 

51 

COLUMN  4: 

1 

15.5 

20 

35793 

95 

65 

Number  of  Blue  Aircraft  Lost  one - 

1 

13.5 

16 

33178 

120 

59 

COLUMN  5: 

1 

13.5 

15 

35129 

96 

58 

Depth  (in  km)  of  Red  Advance 

1 

13.5 

14 

33049 

124 

59 

1 

13.5 

20 

33462 

113 

54 

1 

12.5 

15 

31493 

132 

52 

1 

13.5 

14 

33896 

108 

55 

1 

12.5 

15 

32072 

146 

48 

1 

Finding  the  means  to  each  measure  of  effectiveness 
(means  to  each  vector  of  the  data  set): 


O 

XT 

n 

Xbar  Q  :=  mean(v) 

wi:=xu 

Xbar  j  :=  mean(w) 

$  -  Xi,2 

Xbar  2  :=  mean(g) 

t*r 

ii 

4=r 

Xbar.  3  :=  mean(h) 

m.  :=  X.  . 
i  i,4 

Xbar  4  :=  mean(m) 

E-l 


1 


0 


0 


0 


0 


Define  "Dhalf  ’  as 
the  diagonal  matrix 
with  elements 
1/standard  deviation: 


0.9486833 


2.21735578 


Dhalf  = 


0 


0 


0  0 


0 


0 


0 

1 

1372.90466 

0 

0 


0 

0 

1 

19.7280469 

0 


0 

0 

0 

1 

5.11044682 


Sample  Covariance  Matrix  is: 

S  :=  XT-X  -  — •  (xT-one)- (oneT-x)  r  _  1  c 

i6  C-IT 


13.5 

11.5 

1.71-104  -130.5 

65 

11.5 

73.75 

1 .52- 104  -214.38 

55.75 

s 

= 

1.71-104 

1 .52*  104 

2.83-107  -2.63- 105 

8.76- 104 

-130.5 

-214.38 

-2.63-105  5.84-103  - 

-794.38 

65 

55.75 

8.76- 104  -794.38 

391.75 

0.9 

0.767 

1 . 142*  103  -8.7 

4.333 

0.767 

4.917 

1 .015*  103  -14.292 

3.717 

c  = 

1.142-103 

1.015*  103 

1.885-106  -1.754-104 

5.837-10 

-8.7 

-14.292 

-1.754*  104  389.196 

-52.958 

4.333 

3.717 

5.837*  103  -52.958 

26.117 

E-2 


Here,  we  choose  to  use  the  correlation  matrix,  R,  because  each  vector  uses 
different  units: 


R  :=  DhalfC  Dhalf 


1  0.364  0.877  -0.465  0.894 

0.364  1  0.333  -0.327  0.328 

R=  0.877  0.333  1  -0.648  0.832 

-0.465  -0.327  -0.648  1  -0.525 

0.894  0.328  0.832  -0.525  1 


Eigenvalues  taken  from 
Correlation  matrix,  R: 


evals  =  eigenval^R) 


3.3449 
0.60708 
evals  =  0.07352 
0.82128 
0.15323 


3-345  __  3.345  +  0.82128 

- =  0.669  - =  0.833 

5  5 


66.9%  of  variance  explained  by  first  principal  component. 

(83.3%  of  variance  explained  by  first  two  principal  components.) 


Ar  :=  eigenvecs(R) 
Xd  =  X  -  Xbar 
L  =  Ar  Ahalf 
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The  matrix  of  eigenvectors  is: 


0.504 

0.352 

0.753 

-0.193 

-0.133 

0.276 

0.231 

-0.056 

0.931 

-0.012 

0.514 

-0.024 

-0.499 

-0.185 

-0.672 

-0.393 

0.877 

-0.203 

-0.115 

-0.15 

0.501 

0.231 

-0.373 

-0.219 

0.713 

Principal  Component  Loadings  are  found  in  the  matrix  L: 


eigenval<R)T=(  3.34489892  0.60707727  0.07351713  0.82127912 


0.15322755) 


PCI 

PC2 

PC3 

PC4 

PC5 

0.92191 

0.27395 

0.20421 

-0.17493 

-0.05222 

0.50518 

0.17965 

-0.01517 

0.84396 

-0.00468 

0.9402 

-0.01875 

-0.13539 

-0.16779 

-0.26304 

-0.71817 

0.68335 

-0.05492 

-0.10401 

-0.05859 

0.91653 

0.18011 

-0.10119 

-0.1986 

0.27903 

Days  to  Neutralize  FLOT 
Days  to  Air  Supremacy 
Ending  Red  Inventory 
Blue  Aircraft  Lost 
Depth  of  Enemy  Advance 


Largest  principal  component  correlates  days  to  neutralize  FLOT,  depth  of  enemy 
advance,  ending  Red  inventory,  and  blue  aircraft  lost.  The  first  three  have  an 
opposite  effect  to  Blue  aircraft  lost. 


Second  largest  principal  component  has  a  high  correlation  value  for 
Air  Supremacy  all  alone. 
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Principal  Component  Scores,  Y,  calculated  as: 


Y  =  Xd-DhalfAr 


Principal  Component  Scores: 


2.731  -0.168  -0.228  0.37  0.153 

0.08  0.326  -0.609  1.012  0.208 

2.637  0.19  -0.043  -1.134  0.013 
0.661  -0.024  0.455  0.507  0.315 
-0.632  1.74  -0.072  -1.043  -0.649 
Y=  -1.595  0.561  -0.215  1.472  0.01 
0.004  -1.625  0.116  0.583  -0.506 
-2.406  -0.938  -0.08  -0.326  0.309 
3.507  0.525  0.328  0.484  -0.016 
-0.119  0.254  -0.026  -0.325  0.54 
0.866  -0.997  -0.391  -0.825  -0.367 
-0.496  0.226  0.03  -1.171  0.584 
0.134  0.127  0.207  1.572  -0.265 
-2.332  0.026  0.206  -0.084  0.443 
-0.253  -0.681  0.179  -1.02  -0.267 
-2.786  0.457  0.144  -0.072  -0.504 


Add  center  point  replications  as  additional  points  for  Principal  Component  Scores: 


j  --0..2 


1  14 

17 

33726 

127 

57 

C  := 

13 

16 

33586 

123 

54 

1 13.5 

15 

33813 

122 

56 

Cd  =  C  -  Cbar 
Z  :=  Cd  DhalfAr 


(The  means  used  are  of  the  original  data  set) 
Cbar.  n  :=  mean(v)  Cbar.  ,  :=  mean(g) 

Cbar.  .  :=  mean(w)  Cbar.  -  :=  mean(h) 

Cbar  4  =  mean(m) 


PC  Scores  for  center  points: 


/  0.141  0.754  0.22  -0.035  -0.136 
Z  =  -0.782  -0.031  -0.237  -0.081  -0.309 
\-0.34  0.092  -0.033  -0.713  -0.199 


NOTE:  Center  Point  Replications  are  labeled  as  “x”  on  plot. 
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PrincipalComponent  Score  Plots: 


j  =0.2 


2 


1 

Principal 
Component  4 
(Days  to  0 

Air 

Supremacy) 

-l 


2  -4  -3  -2  -1  0  1  2  3  4 

Principal  Component  1  ("Ground  War  Index") 


Factor  Scores:  After  varimax  rotation,  the  loadings  matrix  appears  as 

FI  F2  F3  F4  F5 

0.94378  -0.17896  0.18532  0.07734  -0.19218' 

Days  to  Neutralize  FLOT 

0.16624  -0.13691  0.97600  0.03218  0.00150  DayS  t0  ^  Supremacy 

0.78655  -0.39841  0.13677  0.45125  -0.01675  Ending  Red  Inventory' 

-0.27956  0.94554  -0.15225  -0.06750  -0.00849  Blue  Aircraft  Lost 

Depth  of  Enemy  Advance 

0.92222  -0.25955  0.14308  -0.02714  0.24683 

eigenvalues:  =  (2.465658  1.170908  1.049284  0.215936  0.098214) 


PC  Scores  with  Center  Points  Included 
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To  standardize  the  data  set,  use: 


Xs  :=  Xd-Dhalf 


Calculating  factor  scores,  use: 

Factor  scores  appear  as: 

Fhat  = 


Center  Points  Factor  Scores: 

Cs  :=  Cd  Dhalf 


Fhat  :=  Xs  R  l  ¥ 


0.957 

-0.865 

0.8 

0.352 

0.935 

-0.333 

0.405 

1.185 

0.682 

2.148 

1.746 

-0.335 

-0.645 

0.351 

0.177 

0.29 

-0.446 

0.575 

-1.535 

-0.965 

0.865 

2.372 

-0.611 

1.405 

-0.725 

-1.093 

0.968 

1.414 

0.127 

0.63 

-1.204 

-1.809 

0.096 

0.911 

-1.026 

-1.371 

-0.471 

-1.026 

-0.618 

0.695 

1.735 

-0.423 

1.219 

-0.375 

-1.034 

0.359 

0.235 

-0.282 

-1.094 

0.832 

-0.007 

-1.067 

-0.958 

1.745 

0.74 

0.557 

0.343 

-1.219 

-1.271 

0.729 

-0.544 

-0.032 

1.648 

0.049 

-1.032 

-0.813 

0.457 

-0.497 

-1.542 

-0.031 

-0.069 

-0.614 

-1.296 

0.305 

-0.893 

-1.077 

1.281 

-0.403 

0.509 

-1.179 

(Note  again  the  means  used  are 
from  the  original  data  set) 


Fhat2  :=  Cs  R  1  F 


Fhat2  = 


0.5 

-0.488 


\  0.129 


0.779  0.205  -0.185  -0.89  \ 
0.308  -0.198  1.06  0.296 

0.304  -0.746  0.497  -0.17 


NOTE:  Center  Point  Replications  labeled  as  squares  on  plots. 
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Factor  Score  Plots: 


FI  vs.  F2  with  Center  Points 


-2  -1.5  -1  -0.5  0  0.5 


Factor  1  ("Ground  War  Index") 


Factor  1  ("Ground  War  Index") 


F2  vs.  F3  with  Center  Points 


Factor  3 
(Days  to  Air 
Supremacy) 


Factor  2  (Blue  Aircraft  Lost) 


APPENDIX  F 


Factor  Analysis  Varimax  Rotation  (SAS  Output) 


FACTOR  ANALYSIS  OF  THUNDER  OUTPUT 


Means  and  Standard  Deviations  from  16  observations 


NEUTRAL 

AIRSUPRM 

REDINVEN 

ATTRIT 

DEA 

Mean 

13.5 

16.625 

33668.4375 

115.4375 

56.625 

Std  Dev 

0.9486833 

2.21735578 

1372.90466 

19.7280469 

5.11044682 

Initial  Factor  Method:  Principal  Components 

Prior  Communality  Estimates:  ONE 
Eigenvalues  of  the  Correlation  Matrix:  Total  =  5  Average  =  1 


1 

2 

3 

4 

5 

Eigenvalue 

3.3449 

0.8213 

0.6071 

0.1532 

0.0735 

Difference 

2.5236 

0.2142 

0.4538 

0.0797 

Proportion 

0.6690 

0.1643 

0.1214 

0.0306 

0.0147 

Cumulative 

0.6690 

0.8332 

0.9547 

0.9853 

1 . 0000 

5 

factors  will  be 

retained  by 

the  NFACTOR 

criterion . 

Factor  Pattern 


FACTOR 1 

FACTOR  2 

FACTOR3 

NEUTRAL 

0.92191 

-0.17493 

0.27395 

days  to  neutralize  FLOT 

AIRSUPRM 

0.50518 

0.84396 

0.17965 

days  to  air  supremacy 

REDINVEN 

0.94020 

-0.16779 

-0.01875 

remaining  red  inventory 

ATTRIT 

“0.71817 

-0.10401 

0.68335 

percent  blue  aircraft  lost 

DEA 

0.91653 

-0.19860 

0.18011 

depth  of  enemy  advance 

FACTOR4 

FACT OR 5 

NEUTRAL 

0.05222 

-0.20421 

days  to 

neutralize  FLOT 

AIRSUPRM 

0.00468 

0.01517 

days  to 

air  supremacy 

REDINVEN 

0.26304 

0.13539 

remaining  red  inventory 

ATTRIT 

0.05859 

0.05492 

percent 

blue  aircraft  lost 

DEA 

-0.27903 

0.10119 

depth  of 

:  enemy  advance 

Variance  explained  by  each  factor 

FACTOR1  FACT0R2  FACT0R3  FACT0R4  FACTOR5 

3.344899  0.821279  0.607077  0.153228  0.073517 


Final  Communality  Estimates:  Total  =  5.000000 

NEUTRAL  AIRSUPRM  REDINVEN  ATTRIT  DEA 

1.000000  1.000000  1.000000  1.000000  1.000000 
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FACTOR  ANALYSIS  OF  THUNDER  OUTPUT 


Rotation  Method:  Varimax 


Orthogonal  Transformation  Matrix 


1 

2 

3 

4 

5 

1 

0.81903 

-0.45612 

0.30882 

0.16007 

0.01201 

2 

-0.37849 

-0.07816 

0.92023 

-0.06049 

-0.01271 

3 

0.40972 

0.87836 

0.23929 

-0.05354 

-0.02209 

4 

-0.10932 

0.08511 

0.00898 

0.82558 

-0.54693 

5 

-0.07815 

0.08419 

0.02174 

0.53504 

0.83670 

Rotated  Factor  Pattern 


FACT OR 1 

FAC  TOR 2 

FACTOR 3 

NEUTRAL 

0.94378 

-0.17896 

0.18532 

days  to 

neutralize  FLOT 

AIRSUPRM 

0.16624 

-0.13691 

0.97600 

days  to 

air  supremacy 

REDINVEN 

0.78655 

-0.39841 

0.13677 

remaining  red  inventory 

ATTRIT 

-0.27956 

0.94554 

-0.15225 

percent 

blue  aircraft  lost 

DEA 

0.92222 

-0.25955 

0.14308 

depth  of 

enemy  advance 

FACTOR4 

FACTORS 

NEUTRAL 

0.07734 

-0.19218 

days  to 

neutralize 

FLOT 

AIRSUPRM 

0.03218 

0.00150 

days  to 

air  supremacy 

REDINVEN 

0.45125 

-0.01675 

remaining  red  inventory 

ATTRIT 

-0.06750 

-0.00849 

percent 

blue  aircraft  lost 

DEA 

-0.02714 

0.24683 

depth  of  enemy  advance 

Variance  explained  by  each  factor 

FACTOR1 

FACTOR2 

FACT  OR  3 

FACTOR4 

FACT OR 5 

2.465658 

1.170908 

1 . 049284 

0.215936 

0.098214 

Final  Communality  Estimates: 

Total  =  5. 

000000 

NEUTRAL 

AIRSUPRM 

REDINVEN 

ATTRIT 

DEA 

1.000000 

1.000000 

1.000000 

1.000000 

1.000000 
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APPENDIX  G  MSPR  and  MAPE  Calculations  using  Microsoft  Excel  5.0 
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APPENDIX  H  Multivariate  Analysis  Validation  Results  using  Mathcad  PLUS  5.0 
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.15  n  :=  0 ..  4 

One  :=  stack(one,one) 

Means  of  each  Measure  of  Effectiveness 

Xbar  „  :=  13.609375 

111,0 

Days  to  Neutralize  FLOT 

Xbar  ,  :=  16.375 
m,  1 

Days  to  Air  Supremacy 

Xbar  0  :=  33650.5 

m,2 

Ending  Red  Inventory 

Xbar  ,:=  114.90625 
m,3 

Blue  Aircraft  Lost 

Xbar  ,  =  56.1875 
m,4 

Depth  of  Enemy  Advance 

Define  the  complete  data  set  as  T:  T  :=  stack(X,  Xval) 

Tbar  :=  stack(Xbar,  Xbar) 
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R  :=  DhalfC  Dhalf 

R  = 


1 

0.068 

0.855 

-0.526 

0.84 

0.068 

1 

0.197 

0.122 

0.187 

0.855 

0.197 

1 

-0.623 

0.791 

-0.526 

0.122 

-0.623 

1 

-0.47 

0.84 

0.187 

0.791 

-0.47 

1 

evals  :=  eigenvaKR) 


evals  = 


3.09548 

0.52026 

0.11004 

1.08653 

0.18768 


3.09548  3.09548  +  1.08653 

- =0.619  - =0.836 

5  5 

61.9%  of  variance  explained  by  first  principal  component. 

(83.6%  of  variance  explained  by  first  two  principal  components.) 

Matrix  of  Eigenvectors  is: 


0.528 

0.348 

0.719 

-0.022 

-0.288 

0.09 

-0.374 

0.154 

0.91 

0.027 

0.537 

-0.028 

-0.611 

0.056 

-0.578 

-0.402 

0.784 

-0.156 

0.394 

-0.209 

0.513 

0.351 

-0.25 

0.114 

0.734 
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Principal  Component  Loadings: 


eigenval<R)T=(  3.09547891  0.52026387  0.11004288  1.08653182  0.18768255) 


PCI 

PC2 

PC3 

PC4 

PCS 

0.92958 

0.25092 

0.2384 

-0.02313  -0.1247 

0.15828 

-0.2696 

0.05095 

0.94844 

0.01159 

0.94466 

-0.02026 

-0.2027 

0.05788  -0.25054 

-0.70727 

0.56566 

-0.05162 

0.41102  -0.09055 

0.90205 

0.25346 

-0.08281 

0.11907 

0.31785 

Days  to  Neutralize  FLOT 
Days  to  Air  Supremacy 
Ending  Red  Inventory 
Blue  Aircraft  Lost 
Depth  of  Enemy  Advance 


Largest  pincipal  component  correlates  days  to  neutralize  FLOT,  depth  of  enemy  advance, 
ending  red  inventory,  and  blue  aircraft  lost.  The  first  three  have  an  opposite  effect  to  Blue 
attrition. 

Second  largest  principal  component  has  a  high  correlation  value  for 
air  supremacy  all  alone. 

(principal  component  scores  calculated  later) 

Factor  Loadings  Matrix  from  varimax  rotation  needed  to  calculate  Factor  Scores: 


rotated  matrix: 

FI  F2 

F3 

F4 

F5 

0.94685  -0.24782 

0.01426 

0.20443 

0.00781 

0.06222  0.06471 

0.99402 

0.05361 

0.03156 

F  := 

0.75261  -0.38507 

0.15004 

0.19940 

0.47225 

-0.28380  0.94483 

0.08733 

-0.11664 

-0.07426 

0.68118  -0.21256 

0.11942 

0.68455 

0.08917 

eigenvalues=  (2.011374  1.151772  1.032674  0.566637 

0.237542  ) 

To  standardize  the  data  set,  use:  Ts  :=  Td-Dhalf 


Principal  Component  Scores,  Y,  calculated  as:  Y  =  Td-Dhalf Ar 

Factor  scores  calculated  using:  Fhat  =  Ts-R  *-F 
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Principal  Component  Scores:  Factor  Scores: 


3.186 

-0.432 

-0.336 

0.688 

0.468" 

"  0.505 

-1.13 

0.961 

1.777 

0.808 
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0.56 
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0.449 
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-1.819 
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1.458 

3.031 
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-0.627 

1.502 

-2.338 

-0.302 

-0.025 

1.245 

0.124 

Fhat  = 

-1.284 

0.707 

1.095 

-0.218 

-0.247 

-0.13 

-2.022 

-0.178 

-0.506 
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0.259 
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-1.075 
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-0.452 
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-0.748 

0.086 

1.689 

-0.3 

-0.042 

-0.139 

1.882 

-0.799 

0.212 

-1.717 
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0.007 

2.265 
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1.781 
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0.34 

0.42 

0.243 

-0.933 

0.071 
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0.02 
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-1.597 

0.407 
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Add  center  point  replications  as  additional  points  for  scores: 


b  :=  0..2 


fu 

17 

33726 

127 

57  \ 

C  := 

13 

16 

33586 

123 

54 

\l3.5 

15 

33813 

122 

56/ 

Cbarb  Q  :=  13.609375 
Cbarb  j  :=  16.375 
Cbarb  2  :=  33650.5 
Cbarb  3  :=  114.90625 

Cbarb  4:=  56.1875 


PC  Scores  for  center  points: 

Cd  :=  C  -  Cbar 
Z  :=  Cd  DhalfAr 


Z  = 


0.146  0.706  0.195  0.519  -0.173\ 
-0.932  -0.04  -0.483  0.002  -0.252 
\-0.235  0.457  -0.326  -0.324  -0.181/ 


Center  Point  Factor  Scores: 
Cs  :=  Cd-Dhalf 

Fhat2  =  Cs  R  1  F 


j  :=  0..  2 


Fhat2  = 


0.815  0.923  0.159  -0.202  -0.366\ 

-0.697  0.399  -0.168  -0.153  1.43 

\-0.019  0.579  -0.618  0.112  0.964/ 
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Principal  Components  Score  Plot 


PC  4 
(Days  to 
Air 

Supremacy) 


PC  Scores  with  Center  Points  Included 


Factor  Score  Plots: 


FI  vs.  F3  with  Center  Points 
T - 1 - 1 - 1 - 1 - 
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Factor  3 
(Days  to 

Air  0 

Supremacy) 


■fr* 


4- 
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Factor  1  ("Ground  War  Index") 


Factor  2 
(Number 
of  Blue 
Aircraft 
Lost) 


F2  vs.  F3  with  Center  Points 


-4  -3  -2-10  1 


Factor  2  (Blue  Aircraft  Lost) 


APPENDIX  I  Factor  Analysis  Varimax  Rotation  of 

Combined  Design  and  Validation  Outputs. 

FACTOR  ANALYSIS  OF  THUNDER  OUTPUT 

Means  and  Standard  Deviations  from  32  observations 

NEUTRAL  AIRSUPRM  REDINVEN  ATTRIT  DEA 

Mean  13.609375  16.375  33650.5  114.90625  56.1875 

Std  Dev  0.78014034  2.53682555  1103.83019  17.1489854  3.93034103 

Initial  Factor  Method:  Principal  Components 

Prior  Communality  Estimates:  ONE 


Eigenvalues  of  the  Correlation  Matrix:  Total  =  5  Average  =  1 


i 

2 

3 

4 

5 

Eigenvalue 

3.0955 

1.0865 

0.5203 

0.1877 

0.1100 

Difference 

2.0089 

0.5663 

0.3326 

0.0776 

Proportion 

0.6191 

0.2173 

0.1041 

0.0375 

0.0220 

Cumulative 

0.6191 

0.8364 

0.9405 

0.9780 

1.0000 

5  factors  will  be  retained  by  the  NFACTOR  criterion. 
Factor  Pattern 


FACTOR  1 

FACTOR2 

FACTOR3 

NEUTRAL 

0.92958 

-0.02313 

0.25092 

days  to  neutralize  FLOT 

AIRSUPRM 

0.15828 

0.94844 

-0.26960 

days  to  air  supremacy 

REDINVEN 

0.94466 

0.05788 

-0.02026 

red  inventory  remaining 

ATTRIT 

-0.70727 

0.41102 

0.56566 

number  of  blue  aircraft 

DEA 

0.90205 

0.11907 

0.25346 

depth  of  enemy  advance 

FACTOR4 

FACTOR 5 

NEUTRAL 

0.12470 

-0.23840 

days  to 

neutralize  FLOT 

AIRSUPRM 

-0.01159 

-0.05095 

days  to 

air  supremacy 

REDINVEN 

0.25054 

0.20270 

red  inventory  remaining 

ATTRIT 

0.09055 

0.05162 

percent 

blue  aircraft  lost 

DEA 

-0.31785 

0.08281 

depth  of  enemy  advance 

Variance  explained  by  < 

sach  factor 

FACTOR  1 

FACTOR2 

FACTOR3 

FACTOR4  FACTOR5 

3.095479 

1.086532 

0.520264 

0.187683  0.110043 

Final  Communality 

Estimates : 

Total  =  5.000000 

NEUTRAL 

AIRSUPRM 

REDINVEN 

ATTRIT  DEA 

1.000000 

1.000000 

1.000000 

1.000000  1.000000 

i-i 


FACTOR  ANALYSIS  OF  THUNDER  OUTPUT 


Rotation  Method:  Varimax 


Orthogonal  Transformation  Matrix 


1 

2 

3 

4 

5 

1 

0.78055 

-0.46645 

0.11574 

0.35112 

0.19103 

2 

0.04154 

0.37537 

0.92149 

0.08396 

0.03422 

3 

0.41840 

0.78566 

-0.36094 

0.26972 

-0.06827 

4 

0.33941 

0.13314 

-0.01170 

-0.81688 

0.44681 

5 

-0.31426 

0.08087 

-0.08391 

0.36003 

0.87065 

Rotated  Factor  Pattern 


FACTOR1 

FACT  OR  2 

FACTOR3 

NEUTRAL 

0.94685 

-0.24782 

0.01426 

days  to 

neutralize  FLOT 

AIRSUPRM 

0.06222 

0.06471 

0.99402 

days  to 

air  supremacy 

REDINVEN 

0.75261 

-0.38507 

0.15004 

red  inventory  remaining 

ATTRIT 

-0.28380 

0.94483 

0.08733 

percent 

blue  aircraft  lost 

DEA 

0.68118 

-0.21256 

0.11942 

depth  of 

enemy  advance 

FACTOR4 

FACTORS 

NEUTRAL 

0.20443 

0.00781 

days  to 

neutralize 

FLOT 

AIRSUPRM 

0.05361 

0.03156 

days  to 

air  supremacy 

REDINVEN 

0.19940 

0.47225 

red  inventory  remaining 

ATTRIT 

-0.11664 

-0.07426 

percent 

blue  aircraft  lost 

DEA 

0.68455 

0.08917 

depth  of  enemy  advance 

Variance  explained  by  each  factor 

FACTOR1 

FACTOR2 

FACTOR3 

FACTOR4 

FACTOR5 

2.011374 

1.151772 

1.032674 

0.566637 

0.237542 

Final  Communality  Estimates: 

Total  =  5. 

000000 

NEUTRAL 

AIRSUPRM 

REDINVEN 

ATTRIT 

DEA 

1.000000 

1.000000 

1.000000 

1.000000 

1.000000 
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